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Design  of  advanced  surveillance  and  tracking  systems  typically  employ  multiple  sensors  which  can  provide 
large  amounts  of  useful  data  to  detect,  identify  and  track  targets  of  interest.  Attempts  at  intelligent  utilization  of  data  for 
optimizing  the  processing  efficiency  in  such  multi-sensor  operations  require  novel  processing  metho  s  w  ic  nee  o 
be  carefullv  tailored  due  to  the  disparate  forms  of  data  and  the  disparity  in  the  resolution  achievable  from  these  sensors. 
This  project  was  primarily  aimed  at  the  super-resolution  processing  of  imagery  data  to  improve  the  resolution  in 
acquired  images  so  that  any  problems  arising  from  the  disparity  in  resolution  levels  can  be  mitigated  and  efficient 
synthesis  of  fiision  mechanisms  can  be  developed.  Specific  investigations  that  were  conducted  as  part  of  this  project 
included:  (i)  A  detailed  analysis  of  resolution  challenges  addressing  several  important  questions  such  as  how  to 
quantify  resolution  in  an  image  (acquired  or  processed),  (ii)  development  of  systematic  digital  processing  algorithms 
obtained  by  employing  a  statistical  optimization  framework  for  achieving  resolution  enhancement  and  super-resolu  ion, 
and  (iii)  performance  evaluation  studies  that  included  results  of  processing  both  simulated  image  data  and  results  ot 
processing  PMMW  data  acquired  from  the  radiometers  being  built  by  the  Air  Force  Wright  Laboratory  Annamen 
Directorate.  The  major  accomplishments  and  research  advances  made  in  this  project  through  rigorous  mathematica 
analysis  and  extensive  simulation  experiments  are  outlined  in  this  report.  Pi  Pi  A  Pi 
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ABSTRACT 

Design  of  advanced  surveillance  and  tracking  systems  typically  employ  multiple  sensors 
which  can  provide  large  amounts  of  useful  data  to  detect,  identify  and  track  targets  of  interest. 
Attempts  at  intelligent  utilization  of  data  for  optimizing  the  processing  efficiency  in  such  multi¬ 
sensor  operations  require  novel  processing  methods  which  need  to  be  carefully  tailored  due  to 
the  disparate  forms  of  data  and  the  disparity  in  the  resolution  achievable  from  these  sensors. 
This  project,  which  was  sponsored  by  the  Air  Force  Office  of  Scientific  Research,  was 
primarily  aimed  at  the  super-resolution  processing  of  imagery  data  to  improve  the  resolution  in 
acquired  images  so  that  any  problems  arising  from  the  disparity  in  resolution  levels  can  be 
mitigated  and  efficient  synthesis  of  fusion  mechanisms  can  be  developed.  The  prmcipal 
objectives  of  this  effort  were  to  develop  and  evaluate  specific  techniques  for  the  restoration  and 
super-resolution  of  images  collected  from  practical  sensing  operations.  In  order  to  increase  the 
relevance  of  this  work  to  the  U.S.  Air  Force  operations,  a  close  liaison  was  maintained  with 
the  Air  Force  Wright  Laboratory  Armament  Directorate.  For  proof  of  concept  demonstrations, 
the  tailoring  of  the  algorithms  as  well  as  the  processing  studies  were  conducted  with  a  special 
focus  on  achieving  resolution  enhancements  in  the  passive  millimeter-wave  (PMMW)  images 
acquired  by  the  Air  Force  Wright  Laboratory  Armament  Directorate  with  the  state-of-the-art 
radiometers  being  built  by  them. 

Specific  investigations  that  were  conducted  as  part  of  this  project  included:  (i)  A 
detailed  analysis  of  resolution  challenges  addressing  several  important  questions  such  as  how  to 
quantify  resolution  in  an  image  (acquired  or  processed),  (ii)  development  of  systematic  digital 
processing  algorithms  obtained  by  employing  a  statistical  optimization  framework  for 
achieving  resolution  enhancement  and  super-resolution,  and  (iii)  performance  evaluation 
studies  that  included  results  of  processing  both  simulated  image  data  and  results  of  processing 
PMMW  data  acquired  from  the  radiometers  being  built  by  the  Air  Force  Wright  Laboratory 
Armament  Directorate.  The  major  accomplishments  and  research  advances  made  in  this  project 
through  rigorous  mathematical  analysis  and  extensive  simulation  experiments  are  outlined  in 
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1.  INTRODUCTION 


1.1  Principal  Objectives  of  the  Project 

A  variety  of  sensing  devices  ranging  from  radar  systems  to  lasers  and  optical  imaging 
systems  are  presently  being  developed  for  facilitating  efficient  target  surveillance  and  tracking 
operations.  The  limitations  of  using  a  single  sensor  in  these  operations,  such  as  limited 
accuracy  and  resolution  and  lack  of  robustness,  have  motivated  the  design  of  surveillance  and 
tracking  systems  with  multiple  sensors  which  can  provide  large  amounts  of  useful  data  to 
detect,  track  and  identify  targets  of  interest.  However,  current  surveillance  and  tracking 
algorithms  usually  use  information  from  only  one  sensor  (such  as  a  track-while-scan  (TWS) 
radar)  or  attempt  to  combine  information  from  different  sensors  in  an  ad  hoc  manner.  While  it 
is  intuitive  that  using  additional  data  available  can  result  in  improved  detection,  classification 
and  track  maintenance  performance,  attempting  to  include  this  data  efficiently  will  require 
novel  processing  methods  which  need  to  be  carefully  tailored  due  to  the  disparate  forms  of  data 
available  from  these  sensors  and  the  disparity  in  the  resolution  of  the  various  sensors.  While 
the  problems  associated  the  former  (viz.,  disparity  of  data  forms)  can  be  overcome  by  the 
development  of  appropriate  feature-level  fusion  algorithms  that  are  based  on  extracting  a  set  of 
key  features  from  each  incoming  data  stream  which  is  then  followed  by  an  integration  of 
features  abstracted  from  the  various  data  streams,  problems  associated  with  the  latter  (viz., 
disparity  in  resolution  levels),  particularly  when  imagery  data  is  involved,  often  complicate  the 
process  of  feature  extraction  and  establishing  correspondence  (  or  registration  )  of  images. 
Consequently  improvement  of  the  resolution  in  the  acquired  image  data  will  almost  always  be 
necessary  before  this  data  can  be  further  used  to  perform  any  decision-making  either 
independently  or  by  fusion  with  other  forms  of  data. 

The  primary  goal  of  this  project  was  the  development  of  advanced  image  restoration 
and  super-resolution  algorithms  for  facilitating  improved  fusion  objectives  m  the  design  of 
efficient  target  surveillance  and  tracking  mechanisms.  An  important  application  area  where 
such  algorithms  play  a  critical  role  is  in  the  design  of  intelligent  integrated  processing 
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architectures  for  achieving  all-weather  day-and-night  terminal  guidance  capabilities  for  smart 
weapons. 

Our  investigations  towards  this  goal  hence  focused  on  several  specific  topics  that 
included  the  following: 

•  tailoring  of  iterative  statistic-based  (Maximum  Likelihood,  Maximum  A  Posteriori) 
super-resolution  algorithms  by  quantifying  a  priori  object  knowledge; 

•  evaluation  of  robustness  of  super-resolution  algorithms  for  imperfect  sensor  models 
and  fluctuations  in  clutter  and  noise  levels; 

•  design  of  blind  deconvolution  algorithms  and  their  implementation  for  optimized 
performance; 

•  evaluation  of  the  role  of  collecting  oversampled  data  in  achieving  reliable  extension 
of  image  bandwidth; 

•  development  of  metrics  to  measure  resolution  in  images  and  to  assess  resolution 
enhancement  in  processed  images. 

Due  to  the  fact  that  investigations  on  resolution  enhancement  have  both  military  and 
non-military  applications,  the  studies  were  conducted  using  a  general  framework  and 
employing  mathematical  models.  However,  in  order  to  provide  validation  of  the  studies 
conducted  and  the  results  obtained  to  military  environments,  the  work  was  performed  in  close 
collaboration  with  the  Air  Force  Wright  Laboratory  Armament  Directorate  personnel.  In 
particular,  the  various  simulation  experiments  conducted  to  demonstrate  the  efficiency  of  the 
algorithms  developed  in  this  project  were  appropriately  tailored  to  process  passive  millimeter- 
wave  (PMMW)  imagery  data  acquired  from  state-of-the-art  PMMW  radiometers  being  built  by 
the  Advanced  Sensors  Group  at  the  Air  Force  Wright  Laboratory  Armament  Directorate. 


1.2  Need  for  Super-resolution  of  Passive  Millimeter-wave  Images 

Present  day  military  operations  that  may  include  execution  of  tactical  surveillance  and 
tracking  missions  in  hostile  environments  often  demand  the  capability  for  all-weather  day-and- 
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night  sensing  and  intelligence  data  collection.  Various  missions  such  as  reconnaissance, 
landing  and  take-off  of  aircraft,  covert  deployment  of  special  operations  teams,  detection  and 
tracking  of  tactical  mobile  and  extended  area  high-value  targets,  and  aimpoint  selection  and 
precision  kill,  are  typical  in  their  need  for  the  ability  to  execute  the  mission  in  any  set  of 
contingencies.  Development  of  sensors  operating  at  different  ranges  of  the  electromagnetic 
spectrum  has  been  the  primary  technological  approach  to  provide  this  capability.  While  two 
types  of  sensing,  viz.,  passive  infra-red  (IR)  and  active  radar,  have  received  a  significantly 
greater  attention  in  the  past,  sensors  operating  in  the  millimeter  wavelength  regime  are  being 
increasingly  considered  in  the  recent  times. 

Passive  Millimeter-wave  (PMMW)  sensing,  in  particular,  combines  the  strong  features 
of  both  IR  and  Synthetic  Aperture  Radar  (SAR)  systems.  Like  IR,  it  provides  a  passive  mode 
of  sensing  and  hence  is  appropriate  for  covert  operations.  Since  measurable  emissions  from  the 
scene  or  target  being  imaged  at  wavelengths  that  are  considerably  longer  than  the  wavelengths 
of  IR  and  visible  light  are  utilized,  significant  all-weather  day-and-night  imaging  capabilities 
are  achieved.  These  wavelengths  are  typically  in  the  range  of  30  to  120  GHz,  corresponding  to 
wavelengths  of  10  mm  to  2.5  mm.  Because  these  are  wavelengths  in  the  microwave  band,  they 
will  pass  through  substantial  amounts  of  attenuating  media,  such  as  fog,  clouds,  rain,  smoke, 
dust,  clothing,  camouflage,  and  even  some  building  material.  Furthermore,  being  a 
fundamental  emission  process,  the  radiation  is  available  not  only  on  bright  sunny  days  but  also 
in  hazy  and  cloudy  environments  and  even  in  the  darkest  of  nights.  Theory  and  background  on 
general  PMMW  radiometry  can  be  readily  found  in  the  literature  [1,2]. 

Although  PMMW  sensing  provides  the  benefits  cited  above  and  hence  is  attractive  for 
all-weather  operations,  there  are  several  disadvantages  the  principal  one  being  poor  resolution 
in  the  acquired  imagery.  The  diffraction-limited  angular  resolution,  0,  of  an  incoherent 
imaging  system  is  given  by  [3] 

X 

0=1.22- 

where  X  is  the  effective  wavelength  of  imaging  and  D  is  the  diameter  of  the  limiting  aperture 
of  the  antenna  or  lens.  As  X  increases,  the  achievable  angular  resolution  decreases,  i.e.  6,  the 
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size  of  the  angle  between  two  resolvable  points,  increases.  For  a  typical  PMMW  sensor  with  a 
1  ft  diameter  antenna  and  operating  at  95  GHz,  the  angular  resolution  is  only  about  10  mrad, 
which  translates  into  a  spatial  resolution  of  about  10  meters  at  a  distance  of  1  Km.  Some  recent 
studies  have  also  established  that  for  ensuring  reasonably  adequate  angular  resolution  (typically 
of  the  order  of  4  mrad),  a  95  GHz  PMMW  imaging  system  with  a  sensor  depression  angle  of 
60°  -  80°  needs  to  be  confined  to  very  low  operational  altitudes  (of  the  order  of  75-100  meters), 
which  puts  inordinate  demands  on  the  surveillance  and  guidance  schemes  to  facilitate  such 
requirements.  In  comparison  with  optical  and  IR  sensors,  since  the  wavelength  of  millimeter- 
wave  radiation  is  1000  times  greater  than  the  wavelength  of  optical  and  IR  radiation,  an 
aperture  of  a  given  diameter  D  has  1000  times  less  resolving  power  than  an  equivalent  optical 
aperture.  Increasing  the  aperture  size  by  1000  times  to  obtain  an  improved  resolution 
comparable  to  that  of  optical  and  IR  sensors  is  usually  infeasible.  This  in  turn  demands  use  of 
clever  image  processing  techniques  to  process  the  acquired  imagery  data  and  obtain  improved 
resolution  in  the  processed  images. 

The  fundamental  problem  underlying  the  sensing  operation  is  the  "low-pass"  filtering 
effect  due  to  the  finite  size  of  the  antenna  or  lens  that  makes  up  the  imaging  system  and  the 
consequent  imposition  of  the  underlying  diffraction  limits.  Hence  the  image  recorded  at  the 
output  of  the  imaging  system  is  a  low-pass  filtered  version  of  the  original  scene.  The  portions 
of  the  scene  that  are  lost  by  the  imaging  system  are  the  fine  details  (high  frequency  spectral 
components)  that  accurately  describe  the  objects  in  the  scene,  which  also  are  critical  for 
reliable  detection  and  classification  of  targets  of  interest  in  the  scene.  Hence  some  form  of 
image  processing  to  restore  the  details  and  improve  the  resolution  of  the  image  will  invariably 
be  needed.  Traditional  image  restoration  procedures  (based  on  deconvolution  and  inverse 
filtering  approaches)  attempt  mainly  at  reconstruction  of  the  passband  and  possibly  elimination 
of  effects  of  additive  noise  components.  These  hence  have  only  limited  resolution  enhancement 
capabilities.  Greater  resolution  improvements  can  only  be  achieved  through  a  class  of  more 
sophisticated  algorithms,  called  super-resolution  algorithms,  which  provide  not  only  passband 
reconstruction  but  also  some  degree  of  spectral  extrapolation,  thus  enabling  to  restore  the 
high  frequency  spatial  amplitude  variations  relating  to  the  spatial  resolution  of  the  sensor  and 
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lost  through  the  filtering  effects  of  the  seeker  antenna  pattern.  A  tactful  utilization  of  the 
imaging  instrument's  characteristics  and  any  a  priori  knowledge  of  the  features  of  the  target 
together  with  an  appropriately  crafted  nonlinear  processing  scheme  is  what  gives  the  capability 
to  these  algorithms  for  super-resolving  the  input  image  by  extrapolating  beyond  the  passband 
range  and  thus  extending  the  image  bandwidth  beyond  the  diffraction  limit  of  the  imaging 

sensor. 


For  application  in  surveillance  environments,  it  must  be  emphasized  that  super¬ 
resolution  is  a  post-processing  operation  applied  to  the  acquired  imagery  and  consequently  is 
much  less  expensive  compared  to  improving  the  imaging  system  for  desired  resolution.  As  an 
example,  it  may  be  noted  that  for  visual  imagery  acquired  from  space-borne  platforms,  some 
studies  indicate  that  the  cost  of  camera  payload  increases  as  the  inverse  2.5  power  of  the 
resolution.  Hence  a  possible  two-fold  improvement  in  resolution  by  super-resolution 
processing  in  this  application  roughly  translates  into  a  reduction  in  the  cost  of  the  sensor  by 
more  than  5  times.  Similar  relations  also  exist  for  sensors  operating  in  other  spectral  ranges 
(due  to  the  relation  between  resolution  and  antenna  size),  confirming  the  cost  effectiveness  of 
employing  super-resolution  algorithms.  The  principal  goal  of  super-resolution  processing  in  a 
multispectral  surveillance  environment  is  hence  to  obtain  an  image  of  a  target  of  interest  via 
post-processing  that  is  equivalent  to  one  acquired  through  a  more  expensive  larger  aperture 

sensor. 


Most  of  the  recent  analytical  work  in  the  development  of  image  restoration  and  super¬ 
resolution  algorithms  has  been  motivated  by  applications  in  Radioastronomy  and  Medical 
Imaging.  While  this  work  has  given  rise  to  some  mathematically  elegant  approaches  and 
powerful  algorithms,  a  certain  degree  of  care  should  be  exercised  in  adapting  these  approaches 
and  algorithms  to  tactical  surveillance  scenarios.  This  is  due  to  the  convergence  problems  often 
encountered  by  iterative  schemes  and  the  specific  statistical  models  representing  the  scenarios 
facilitating  their  development.  For  example,  a  slowly  converging  algorithm  that  ultimately 
guarantees  the  best  resolution  in  the  processed  image  may  pose  no  implementational  problems 
in  radioastronomy;  however,  it  could  be  entirely  unrealistic  for  implementation  in  an 
autonomous  unmanned  tactical  system  that  must  operate  fast  enough  to  track  target  motion. 
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Hence  a  careful  tailoring  of  the  processing  algorithm  is  of  critical  importance  in  order  to 
realize  the  possible  performance  benefits  from  super-resolution  processing  which  include  better 
false  target  rejection,  improved  automatic  target  recognition  and  reliable  tracking. 

1 .3  Outline  of  the  Report 

The  principal  objectives  in  this  project  are  the  development  of  novel  super-resolution 
algorithms  for  processing  PMMW  imagery  data  in  order  to  achieve  resolution  enhancements 
that  facilitate  improved  surveillance  and  tracking  of  targets  of  interest.  In  this  report,  we  shall 
describe  the  various  studies  conducted  and  the  major  results  obtained  towards  meeting  the 

project  goals. 

In  Section  2,  some  background  material  that  is  essential  for  understanding  the 
significance  of  the  tools  employed  in  this  study  will  be  briefly  reviewed.  A  mathematical 
description  of  the  objectives  in  designing  restoration  and  super-resolution  algorithms  will  be 
given  which  provides  a  model-based  framework  for  the  analytical  development  of  restoration 
schemes.  The  role  of  using  a  priori  knowledge  in  the  design  of  algorithms  that  provide  true 
super-resolution,  i.e.  spectral  extrapolation  beyond  the  image  passband,  will  be  given  a 
particular  focus.  Some  practical  issues  that  need  to  be  taken  into  account  in  the  design  of 
satisfactory  algorithms  will  be  described.  Some  metrics  that  can  be  used  to  assess  the  image 
quality  and  the  resolution  enhancement  subsequent  to  super-resolution  processing  will  also  be 

introduced. 

In  Section  3  we  shall  give  a  detailed  derivation  of  an  iterative  super-resolution 
algorithm  obtained  from  employing  a  Maximum  Likelihood  (ML)  approach  and  describe  its 
restoration  and  super-resolution  performance.  Since  computational  complexity  is  of  particular 
concern  in  the  practical  implementation  of  iterative  restoration  algorithms,  we  will  provide  an 
analysis  of  processor  requirements  for  implementing  the  ML  algorithm.  Results  of  processing  a 
number  of  Passive  Millimeter-wave  (PMMW)  images  acquired  from  state-of-the-art 
radiometers  will  also  be  included  in  this  section. 
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Section  4  describes  a  number  of  modified  versions  of  the  basic  ML  algorithm  aimed  at 
obtaining  optimal  performance  even  under  some  non-ideal  conditions  of  operation.  Since  in 
practical  imaging  operations,  the  parameters  of  the  imaging  system  cannot  be  assumed  to  be 
perfectly  known,  a  modified  algorithm  that  iteratively  updates  the  sensor  point  spread  function 
in  addition  to  performing  an  iterative  estimation  of  the  restored  image  pixels  will  be  developed. 
For  use  in  environments  where  the  signal-to-noise  ratio  (SNR)  is  rather  poor,  some 
modifications  that  attempt  to  suppress  noise-induced  processing  artifacts  will  be  outlined.  For 
suppression  of  ringing  artifacts  in  the  processed  image,  another  modification  that  incorporates 
a  background-detail  separation  approach  in  the  estimation  process  will  be  described.  Yet 
another  modified  version  which  is  obtained  by  maximizing  the  posterior  distribution  function 
and  which  yields  a  Maximum  A  Posteriori  (MAP)  restoration  algorithm  will  also  be  described 
in  this  section.  We  shall  also  present  some  processing  results  obtained  by  employing  these 
modified  algorithms  in  order  to  illustrate  their  strong  performance  features. 

The  role  of  oversampled  data  in  the  resolution  enhancement  performance  will  be  given 
a  particular  focus  in  Section  5  and  a  new  progressive  upsampling  procedure  that  provides 
optimized  implementations  of  iterative  super-resolution  algorithms  will  be  described.  An 
illustration  of  the  performance  of  this  procedure  will  also  be  given  in  this  section  by  processing 
a  few  representative  images. 

Finally  in  Section  6,  we  shall  present  a  summary  of  the  results  accomplished  during 
this  project  and  offer  some  concluding  remarks.  An  outline  of  some  useful  research  directions 
for  extending  work  on  this  project  will  also  be  given.  A  complete  listing  of  the  programs  in  C 
language  that  implement  the  algorithms  presented  here  will  be  given  in  an  appendix  at  the  end 

of  the  report. 
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2.  FACTORS  THAT  INFLUENCE  OPTIMAL  DESIGN  OF 
SUPER-RESOLUTION  ALGORITHMS 

2.1  Mathematical  Description  of  Objectives  in  Image  Restoration  and  Super-resolution 

Development  of  efficient  methods  for  restoration  of  degraded  images  has  a  considerably 
long  history  [4]  and  a  number  of  algorithms  derived  by  employing  deterministic  and  stochastic 
frameworks,  some  of  which  are  even  suitable  for  digital  implementation,  are  in  existence.  In 
general,  these  methods  attempt  to  recover  the  features  of  an  object  or  a  scene  that  is  imaged  m 
an  imperfect  manner.  The  objectives  in  the  restoration  processing  could  range  from  simple 
noise  removal  (by  appropriate  filtering  operations)  to  more  elaborate  processing  that  attempts 
to  reverse  the  blur  in  the  acquired  image  which  may  be  caused  by  a  number  of  factors  such  as 
relative  motion  between  the  object  and  the  imaging  sensor  and  out-of-focus  imaging.  While 
these  methods  generally  result  in  noticeable  improvements  in  the  resolution  present  in  the 
processed  image,  the  primary  emphasis  in  the  processing  is  on  passband  reconstruction  only, 
i.e.,  restoring  the  frequency  components  within  the  bandwidth  of  the  sensor.  In  contrast, 
super-resolution  is  a  form  of  image  restoration  where  the  primary  goal  is  to  recover  attenuated 
frequencies  from  diffraction  limited  imaging  operations  by  appropriately  extrapolating  the 
image  bandwidth  beyond  the  sensor  cutoff  frequency.  Thus  the  primary  objective  of  super¬ 
resolution  processing  is  to  obtain  a  processed  image  that  is  equivalent  to  one  obtained  from  a 
larger  aperture  (higher  cost)  sensor.  A  more  detailed  description  of  the  objectives  of  image 
restoration  and  super-resolution  from  a  frequency  spectrum  reconstruction  viewpoint  can  be 

found  in  [5]. 

For  a  precise  description  of  the  restoration  and  super-resolution  objectives,  it  is  useful 
to  start  with  a  mathematical  model  for  the  image  formation  process.  In  particular,  for 
obtaining  a  discrete  model  of  the  imaging  process  to  serve  as  a  basis  for  tailoring  a 
restoration/super-resolution  algorithm,  it  is  common  to  start  with  the  assumption  that  the 
blurred  image  can  be  represented  as  the  output  of  a  linear  shift-invariant  system  with  a  finite 
extent  Point  Spread  Function  (PSF)  whose  input  is  the  scene  (or  object)  that  is  imaged.  Hence, 
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considering  that  the  image  is  obtained  from  an  incoherent  sensing  operation  and  any  noise 
contamination  can  be  approximated  by  an  additive  zero-mean  white  Gaussian  random  field 
which  is  independent  of  the  object,  one  can  obtain  a  model  for  the  degraded  image  in  the  form 

g(x,^)=  £a(x-*o>-50/(*»50  +  K*O')  (1) 

where  f{x,y)  denotes  the  object’s  intensity  function  defined  over  a  region  (x ,y)eR0 , 
denotes  the  intensity  detected  in  the  image  over  a  region  ( x,y)sR , ,  h(zx,z2)  denotes  the  sensor 
PSF,  and  n{x,y)  denotes  the  additive  noise  contamination.  The  classical  image  restoration 

problem  is  to  find  the  object  intensity  estimate  {/(*,>>)}  given  the  image  data  {g(*,.y)}  and 

knowledge  of  the  PSF  function  h{zx,z2)  for  the  imaging  sensor. 

Assuming  that  the  image  to  be  processed  consists  of  Mx  M  equally  spaced  gray  level 
pixels  obtained  through  a  sampling  of  the  image  field  at  a  rate  that  satisfies  the  Nyquist 
criterion,  and  using  a  lexicographical  ordering  of  the  signals  g ,  /  and  n ,  one  can  rewrite 

Equation  (1)  as  a  convolution  of  two  one-dimensional  vectors  h  =  [/i(l),/i(2),...,/i(7V)]  and 
/  =  [/(l),/(2),...,/(iV)]rin  the  form 

g(i)  =  h(i)  ®  /(/)  +  n(i) 

n  (2) 

=  tlh(i-j)f(j)  +  n(  *)  ’  i  =  1’2’-’N 

y=i 

where  N  =  M2  and  ®  denotes  convolution.  More  compactly,  Equation  (2)  can  be  rewritten  as 
the  vector  equation 


g=Hf  +  n 


(3) 


where  g ,  / ,  and  n  are  vectors  of  dimension  N ,  and  H  denotes  the  PSF  block  matrix  whose 
elements  can  be  constructed  from  the  PSF  samples  {h(\),h(2),-,h{N)}  •  11  should  be  noted  that 
Equations  (2)  and  (3)  represent  space-domain  models  and  are  equivalent  to  the  frequency- 
domain  model 


G{a>)  =  H{co)F(co)  +  N(o)) 


(4) 


where  co  is  the  discrete  frequency  variable,  and  G{co) ,  F{co) ,  H(co) ,  and  N{co)  are  the 
Discrete  Fourier  Transforms  (DFT’s)  of  the  N  -point  sequences  g(i) ,  /O'),  MO.  and  n(i) 
respectively. 


Diffraction  limits  underlying  practical  sensing  operations  result  in  acquiring  blurred 
images  with  inherently  poor  resolution.  The  blurring  effect  caused  by  the  sensor  results  in  the 
image  spectrum  G{co)  being  a  low  pass  filtered  version  of  the  object  spectrum  F{a>) ,  and 
consequently  the  higher  frequencies  present  in  the  object  (which  are  responsible  for  the  spatial 
resolution)  will  be  attenuated  at  the  cutoff  frequency  coc  determined  by  the  diffraction  limit  of 


the  sensor.  The  primary  goal  of  super-resolution  processing  is  to  extend  the  image  bandwidth 
beyond  coc  by  estimating  these  attenuated  spatial  frequencies  in  the  range  coc<co<coj2 , 
where  cos  is  the  sampling  frequency  used.  An  efficient  super-resolution  algorithm  effectively 
utilizes  available  knowledge  of  the  imaging  sensor’s  characteristics  and  any  a  priori  knowledge 


a  T  a  a  A  /  \ 

of  the  scene  being  imaged  to  construct  the  object  estimates  /  = 


such 


that 


/  =  arg  min7  j{g,f)  =  K£  minf 


(5) 


where  j(g,f)  is  an  appropriately  chosen  norm  to  measure  the  closeness  of  the  estimate  to  the 
original  object.  It  must  be  emphasized  that  the  bandwidth  extension  beyond  coc  serves  as  a 
measure  of  the  efficiency  of  the  super-resolution  algorithm  tailored.  One  may  also  note  in  this 
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context  that  traditional  image  restoration  procedures  which  attempt  to  implement  a 
convolutional  inverse  (as  for  instance,  by  computing  the  inverse  of  the  PSF  matrix  H  )  are 
generally  successful  in  correcting  the  degradations  within  the  passband  only,  i.e.  in 
0  <co<coc,  while  having  no  significant  bandwidth  extension  capabilities. 

2.2  Limits  on  Super-resolution 

The  idea  of  recreating  the  spectral  components  that  are  removed  by  the  imaging  process 
and  hence  are  not  present  in  the  image  available  for  processing  may  pose  some  conceptual 
difficulties,  which  may  lead  one  to  suspect  whether  super-resolution  is  indeed  possible. 
Fortunately  there  exist  sound  mathematical  arguments  confirming  the  possibility  of  spectral 
extrapolation.  The  primary  justification  comes  from  the  Analytic  Continuation  Theorem  and 
the  property  that  when  an  object  has  finite  spatial  extent  its  frequency  spectrum  is  analytic  [6]. 
Due  to  the  property  that  a  finite  segment  of  any  analytic  function  in  principle  determines  the 
whole  function  uniquely,  it  can  be  readily  proved  that  knowledge  of  the  passband  spectrum  of 
the  object  allows  a  unique  continuation  of  the  spectrum  beyond  the  diffraction  limit  imposed  by 
the  imaging  system.  It  must  be  emphasized  that  the  limited  spatial  extent  of  the  object  is  critical 
in  providing  this  capability  for  extrapolation  in  the  frequency  domain. 

The  Gerchberg-Papoulis  formalism  [7,8]  identifies  a  possible  iterative  procedure  for 
solving  the  super-resolution  problem.  This  algorithm  alternately  applies  constraints  in  the 
space-domain  and  the  frequency-domain  in  the  quest  for  reconstructing  the  unknown  portion  of 
the  frequency  spectrum  from  a  limited  known  portion,  viz.  the  passband  of  the  sensor,  when  a 
priori  knowledge  of  the  spatial  extent  of  the  object  is  available  (the  object  values  within  this 
extent  are  not  known,  however).  Each  iteration  of  the  algorithm  is  a  two-step  procedure  that 
can  be  summarized  as  under: 

STEP  1—  Transform  known  passband  spectrum  to  space-domain  and  apply  space  limit 
constraint  to  set  object  values  to  zero  outside  the  region  where  it  is  known  to 
be  space  limited. 

STEP  2—  Transform  result  back  to  frequency-domain  and  correct  the  passband  back  to 
the  known  original  values. 
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These  two  steps  are  iteratively  applied  until  a  satisfactory  estimate  of  the  spectral  components 
being  reconstructed  is  obtained.  Analytical  support  for  the  convergence  of  the  procedure  comes 
from  consideration  of  the  energy  of  the  error  function  (error  between  the  true  spectrum  and  its 
estimate).  The  application  of  constraints  in  each  iteration,  once  in  the  space-domain  and  the 
other  in  the  frequency-domain,  provides  the  nonlinear  operations  to  generate  the  new 
frequencies  needed  for  super-resolution. 

The  arguments  given  above  clearly  attest  to  the  possibility  of  spectral  extrapolation 
beyond  the  sensor  cutoff  frequency  coc .  The  question  then  is  to  determine  whether  there  exist 
any  limits  on  the  bandwidth  extension  possible  or  whether  it  is  possible  to  reconstruct  the 
entire  spectrum  (up  to  the  limit  imposed  by  the  sampling  rate  used,  viz.  coJ2).  Evidently,  the 
amount  and  the  quality  of  bandwidth  extension  are  measures  of  the  performance  of  a  super¬ 
resolution  algorithm. 

The  development  of  limits  for  signal  recovery  has  been  of  interest  in  Communications 
and  Information  Sciences  for  a  number  of  years  (Shannon's  information  transfer  limits).  In  the 
present  context,  it  is  intuitive  to  expect  that  the  limits  on  spectrum  extrapolation  come  from  a 
number  of  factors,  most  notably  those  that  affect  the  precision  of  the  data  being  processed. 
Clearly,  one  of  the  important  factors  is  the  noise  level  present,  i.e.  SNR  of  the  image.  The  role 
of  SNR  in  affecting  the  quality  of  image  restoration  has  long  been  studied.  In  fact,  many  of  the 
available  approaches  for  image  restoration  start  from  recognizing  the  problem  associated  with 
inverting  the  imaging  equation  described  earlier  in  Equation  (3),  viz., 


g  =  Hf  +  n , 

as  an  ill-posed  problem,  i.e.  the  object  estimate  f  obtained  from  inverting  the  above  equation 
is  not  a  continuous  function  of  image  data  g  and  arbitrarily  small  perturbations  in  g  can  lead  to 

large  variations  in  the  estimate  / ,  and  attempting  to  devise  procedures  (such  as  regularization 
[9,10])  for  overcoming  this  problem. 
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Several  other  factors  can  equally  influence  any  limits  on  possible  spectrum 
extrapolation  and  consequently  establishing  precise  analytical  limits  to  the  bandwidth  extension 
possible  in  all  cases  seems  infeasible.  For  optical  images,  by  using  an  information  theoretic 
analysis,  Kosarev  [11]  determined  a  resolution  limit  under  certain  conditions  to  be 


1  _  (6) 

a>c\og2(\  +  SNR) 


where  r,  is  the  minimum  resolvable  distance  between  two  point  sources  in  the  object,  is  the 
cutoff  frequency  of  the  imaging  system  and  SNR  is  the  image  signal-to-noise  ratio.  It  may  be 
noted  that  the  quantity  on  the  right-hand-side  is  fixed  for  any  specific  image  data  and  hence  any 
separation  greater  than  the  value  of  r,  can  be  potentially  resolved  by  super-resolution 
processing.  This  formula  also  specifies  the  role  of  *>c  in  obtaining  resolution  limits.  Note  that 
as  c oc  increases,  r,  decreases  (i.e.  resolution  increases)  which  agrees  well  with  intuition. 

Another  term  that  affects  any  limits  on  spectrum  extrapolation  is  the  sampling  rate 
used.  Clearly,  higher  the  value  of  larger  will  be  the  width  of  the  frequency  band 
(qc  <  co  <  co J2  for  any  imaging  system,  and  greater  is  the  potential  for  resolution 
improvement.  In  fact,  many  practical  super-resolution  algorithms  use  interpolation  or  other 
upsampling  procedures  appropriately  during  execution  of  the  algorithm  to  obtain  the  effect  of 

larger  cos  [12]. 

One  factor  that  is  not  readily  apparent  in  regard  to  its  relation  to  resolution  limits  is  the 
spatial  extent  of  the  object.  Very  recently,  Sementilli  et.al.  [13]  have  derived  an  approximate 
bound  on  bandwidth  extrapolation  for  optical  images  which  involves  ©c,  ©,  and  the  object 
extent.  This  bound  not  only  establishes  that  the  potential  for  super-resolution  increases  with 
decreasing  spatial  extent  of  objects  being  imaged,  but  also  confirms  the  intuitive  feeling  that 
restoration  results  often  demonstrated  for  two-point  source  objects  do  not  necessarily  translate 
into  corresponding  performances  in  the  case  of  spatially  extended  objects. 
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It  should  be  emphasized  that  while  analytical  limits  such  as  the  ones  discussed  above 
are  useful  for  determining  potential  benefits  from  super-resolution  processing  of  given  image 
data,  one  does  not  have  to  be  necessarily  discouraged  if  the  data  to  be  processed  does  not  meet 
these  limits.  Observe  that  only  the  characteristics  of  the  image  data  g  and  of  the  imaging 
system,  and  only  a  limited  information  on  the  object  (viz.,  spatial  extent)  are  used  in 
developing  these  limits.  If  any  additional  knowledge  of  the  object  being  restored  is  available, 
one  may  attempt  to  use  this  information  for  possible  spectral  extrapolation  and  hence  improved 
resolution.  How  to  utilize  this  information  is  at  the  heart  of  a  well-tailored  super-resolution 
procedure. 

2.3  Use  of  a  priori  Knowledge  in  Design  of  Processing  Schemes 

As  noted  in  the  last  section,  due  to  the  ill-posed  nature  of  the  inverse  filtering  problem 
underlying  image  restoration  and  super-resolution  objectives,  it  is  necessary  to  have  some  a 
priori  information  about  the  ideal  solution,  i.e.  the  object /being  restored  from  its  image  g.  In 
algorithm  development,  this  information  is  used  in  defining  appropriate  constraints  on  the 
solution  and/or  in  defining  a  criterion  for  the  "goodness"  of  the  solution.  It  may  be  recalled 
that  the  use  of  such  constraints  is  fundamental  in  the  application  of  Gerchberg-Papoulis 
formalism  and  is  in  fact  the  basis  for  the  nonlinear  processing  underlying  super-resolution. 


The  specific  a  priori  knowledge  that  can  be  used  evidently  depends  on  the  specific 
application.  For  applications  in  astronomy,  it  could  come  in  the  form  of  some  known  facts 
about  the  spectral  differences  of  the  objects  one  is  looking  for.  In  medical  imaging  and  in 
military  applications,  it  could  come  from  the  geometrical  features  of  the  object  (target  shape, 
for  instance).  Also,  one  could  use  the  fundamental  knowledge  that  the  reflectivity  of  any  point 
on  the  ground  cannot  be  negative.  In  addition  to  the  nonnegativity  constraint,  a  space 
constraint  resulting  from  the  known  space-domain  limits  on  the  object  of  interest  could  be 
used.  Other  typically  available  constraints  include  level  constraints  (which  impose  upper  and 
lower  bounds  on  the  intensity  estimates  /,  )  ,  smoothness  constraints  (which  force  neighboring 
pixels  in  the  restored  image  to  have  similar  intensity  values)  and  edge-preserving  constraints. 
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More  complicated  constraints  are  possible,  but  in  general  they  result  in  tuning  the  algorithms 
to  specific  classes  of  targets. 


Varying  by  the  extent  to  which  a  priori  knowledge  can  be  incorporated  in  algorithm 
development,  there  have  been  introduced  into  the  literature  a  large  number  of  image  restoration 
approaches  and  algorithms  too  vast  to  describe  or  reference  here.  One  may  refer  to  some 
recent  survey  papers  [14,15]  for  a  review  of  the  extensive  activity  on  this  topic.  In  the  context 
of  the  objectives  of  the  present  project  however,  it  is  important  to  recognize  that  only  a  small 
subset  of  the  approaches  that  are  developed  for  image  restoration  have  received  some  interest 
for  their  super-resolution  capabilities,  i.e.  possible  spectrum  extrapolation  performance.  One 
may  note  that  not  all  image  restoration  methods  provide  the  capability  for  super-resolving.  In 
fact,  a  majority  of  existing  schemes  may  perform  decent  passband  restoration,  but  provide  no 
bandwidth  extension  at  all. 

The  various  approaches  in  general  attempt  to  code  the  a  priori  knowledge  to  be  used  by 
specifying  an  object  model  or  a  set  of  constraint  functions,  and  further  employ  an  appropriate 
optimization  criterion  to  guide  in  the  search  for  the  best  estimate  of  the  object.  A  convenient 
way  of  classifying  the  resulting  algorithms  is  into  iterative  and  non-iterative  (or  direct) 
schemes.  Figs,  la  and  lb  depict  schematically  these  two  basic  approaches.  Non-iterative 
approaches  generally  attempt  to  implement  an  inverse  filtering  operation  (without  necessarily 
performing  the  computation  of  the  inverse  of  the  PSF  matrix  H,  however)  and  have  poor  noise 
characteristics.  All  required  computations  and  any  possible  use  of  constraint  functions  are 
applied  in  one  step.  In  contrast,  iterative  methods  apply  the  constraints  in  a  distributed  fashion 
as  the  solution  progresses  and  hence  the  computations  at  each  iteration  will  be  generally  less 
intensive  than  the  single-step  computation  of  non-iterative  approaches.  Some  additional 
advantages  of  iterative  techniques  are  that,  (1)  they  are  more  robust  to  errors  in  the  modeling 
of  the  image  formation  process  (uncertainties  in  the  elements  of  the  PSF  matrix  H,  for 
instance),  (2)  the  solution  process  can  be  better  monitored  as  it  progresses,  (3)  constraints  can 
be  utilized  to  better  control  the  effects  of  noise  (and  possibly  clutter),  and  (4)  can  be  tailored  to 
offset  sensor  nonlinearities.  The  disadvantages  of  these  methods  generally  are,  (1)  mcreased 
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computation  time  if  a  number  of  iterations  are  to  be  executed  for  obtaining  the  required  level  of 
resolution  enhancement,  and  (2)  need  for  proving  convergence  of  the  iterative  scheme.  Despite 
these  disadvantages,  iterative  methods  are  generally  the  preferred  approach  due  to  their 
numerous  advantages  and  also  since  the  iteration  can  be  terminated  once  a  solution  of  a 
reasonable  quality  is  achieved.  Consequently,  most  of  the  research  in  tailoring  super-resolution 
algorithms  is  taking  place  at  present  times  in  formulating  iterative  procedures. 


A  priori  knowledge  and  constraints 


Object  imaged  / 


Sensor 


Super-resolution  Filter 


Image  g 

Fig.  la.  Schematic  of  Noniterative  ( Direct)  Super-resolution 


Object  estimate  f 


A  priori  knowledge  and  constraints 

Fig.  lb.  Schematic  of  Iterative  Super-resolution 
Fig.  1.  Iterative  and  Non-iterative  Super-resolution  Schemes 


2.4  Some  Issues  of  Particular  Concern  in  Design  of  Super-resolution  Algorithms 

An  important  consequence  of  the  ill-posed  nature  of  the  deconvolution  problem 
underlying  image  restoration  and  super-resolution  processing  is  the  generation  of  artifacts  in 
the  processed  image.  Restoration  artifacts  are  patterns  and/or  geometrical  structures,  such  as 
lines,  points,  etc.,  that  are  formed  in  the  restored  image  which  are  not  present  in  the  object 
imaged.  Several  phenomena  contribute  to  the  production  of  artifacts  in  the  restoration  process 
[16]  and  the  intensity  of  these  can  magnify  as  the  number  of  iterations  is  increased.  Presence  of 
artifacts  in  the  processed  image  is  generally  undesirable  since  it  can  interfere  with  the  overall 
goals  for  which  restoration  and  super-resolution  processing  are  included,  such  as  reliable  target 
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detection  and  recognition,  due  to  the  fact  that  the  artifacts  could  obscure  some  critical  feature 
of  the  target,  introduce  features  that  may  not  be  present  in  the  target  of  interest,  or  otherwise 
cause  distortions  of  feature  maps  that  may  be  extracted  from  the  restored  image.  Consequently, 
some  type  of  post-filtering  and/or  modified  implementations  will  generally  be  needed  for 
eliminating  or  suppressing  the  artifacts  as  they  are  generated. 

An  important  factor  that  could  impose  limits  on  successful  restorations  is  the  noise  level 
present  (i.e.  SNR  of  the  image).  As  discussed  earlier,  Kosarev’s  [11]  limit  to  the  achievable 
spectral  extrapolation  inversely  relates  the  minimum  resolvable  distance  between  two  point 
sources  in  the  scene  imaged  and  the  SNR  of  the  obtained  image.  Although  artifact  production 
during  image  restoration  is  due  to  a  number  of  factors,  the  presence  of  significant  amounts  of 
noise  in  the  measured  data  (poor  SNR  levels)  is  perhaps  a  major  contributing  factor  [17]. 
Furthermore,  in  the  super-resolution  processing  of  images  that  contain  high  frequency  noise, 
the  creation  of  higher  frequencies  in  the  processed  image  could  be  a  resultant  of  energy 
flowing  from  noise  into  the  image  pixels,  which  evidently  needs  to  be  curtailed.  Consequently, 
the  dynamics  of  information  transfer  between  the  noise  present  and  the  image  during 
successive  iterations  are  of  particular  importance,  and  any  modifications  to  the  iterative 
algorithm  that  take  into  account  these  dynamics  are  of  specific  interest.  This  is  of  particular 
practical  significance  in  the  processing  of  several  sensor-acquired  imagery  data,  such  as 
PMMW  images,  since  typical  radiometric  images  are  characterized  by  SNR  levels  which  are 
about  20  dB  or  less . 

The  artifacts  generated  due  to  the  presence  of  noise  in  the  image  data  manifest 
themselves  in  the  restored  image  much  differently  from  the  more  familiar  “ringing  artifacts” 
[18].  The  latter  are  primarily  due  to  Gibbs  phenomena  resulting  from  the  truncation  of  Fourier 
components  caused  by  the  extent  of  the  acquired  spatial  frequencies  of  the  scene  or  the  object 
imaged  being  finite.  Consequently,  during  the  restoration  process,  particularly  when  statistical 
estimation  methods  are  employed,  noticeable  distortions  near  edges  occur  as  sharp  transitions, 
or  edges  that  may  be  present  in  the  intensity  distribution  function  become  more  accentuated 
and  emerge  with  overshoots  (or  ringing).  These  artifacts  show  up  in  the  processed  image  as 
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periodic  (or  decaying  periodic)  repetitions  of  sharp  intensity  transitions  and  generally  become 
more  intense  as  the  number  of  iterations  in  super-resolution  processing  grows. 

Commonly  used  techniques  for  eliminating  restoration  artifacts  employ  some  form  of 
filtering  of  the  processed  image.  While  these  can  reduce  the  artifacts,  they  also  generally 
reduce  the  resolution,  counteracting  any  gains  produced  from  super-resolution  processing. 
Consequently,  clever  techniques  need  to  be  employed  for  achieving  a  reduction  of  the  artifacts 
without  the  disadvantage  of  resolution  loss.  Two  such  methods  that  involve  a  post-filtering 
adjustment  technique  and  a  background-detail  separation  of  the  image  to  process  them 
separately  will  be  described  later  in  this  report. 

Yet  another  cause  for  artifact  production  that  requires  attention,  particularly  in  super¬ 
resolution  processing,  is  the  aliasing  of  the  extrapolated  spectral  components  with  the  original 
frequency  components  present  in  the  passband  spectrum  of  the  image  data.  This  is  usually  the 
result  of  an  insufficient  rate  of  sampling  used  for  data  collection  and  requires  some 
sophisticated  processing  steps  to  eliminate.  Since  inclusion  of  these  processing  steps  is 
fundamental  to  achieving  the  goals  of  super-resolution,  this  issue  will  be  discussed  in  a  greater 
detail  in  a  later  section. 

Although  general  purpose  algorithms  for  image  restoration  and  super-resolution  can  be 
developed  without  using  excessive  knowledge  of  the  sensor  design  and  the  variations  in  the 
sensing  process  at  different  wavelengths,  some  understanding  of  the  sensor  phenomenology  can 
be  utilized  in  an  attempt  to  optimize  the  processing  algorithm  for  the  specific  imaging 
modality.  It  is  well  known  that  the  contrast  behavior  of  an  object  that  is  imaged  against  its 
background  depends  not  only  on  the  sensor  parameters  (such  as  wavelength  of  radiation, 
detector  polarization  and  thermal  sensitivity),  but  also  on  the  object  characteristics  (such  as 
reflectivity,  physical  and  chemical  composition,  shape  and  physical  extent)  and  also  on 
atmospheric  conditions.  Thus,  man-made  and  natural  objects  appear  with  different  brightness 
in  the  image.  For  example,  in  PMMW  images,  vegetation  and  bare  soil  may  appear  brighter 
due  to  their  warm  physical  temperature,  whereas  metal  objects  on  flat  earth  reflect  the  “cold” 
sky  and  hence  appear  in  the  image  with  lower  brightness. 
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Some  additional  considerations  are  useful  in  tailoring  super-resolution  algorithms  for 
PMMW  images.  In  comparison  with  the  contrast  available  in  IR  imagery,  one  may  note  that 
black  body  radiation  in  the  MMW  portion  of  the  spectrum  is  much  more  dependent  on  the 
emissivity /reflectivity  of  the  object  of  interest  than  on  the  temperature  of  the  object.  In  the 
MMW  range,  the  emitted  energy  is  a  function  of  T,  the  object  temperature,  rather  than  on 
T4  as  it  is  in  the  IR  range  of  the  spectrum  [12].  Furthermore,  at  MMW  wavelengths,  black 
body  radiation  can  be  described  by  the  Rayleigh-Jeans  law  (which  provides  a  linear 
approximation  to  Planck’s  law)  in  the  form 


Nbb  = 


2  ckT 

JA  " 


(7) 


where  c  is  the  speed  of  light,  k  is  Boltzmann’s  constant,  T  is  the  object’s  temperature  (in 

2 ck 

Kelvin)  and  /l  is  the  wavelength.  Thus  denoting  the  proportionality  constant  =  C ,  one 


can  invert  this  relationship  to  obtain  an  expression  for  the  “equivalent  black  body  temperature” 
of  an  object  for  a  given  radiance  level  received  at  the  aperture  of  the  sensor  as 

T  =  (8) 

e*  C  ' 

Teq  is  usually  referred  to  as  “radiometric  temperature”  in  MMW  radiometry.  Typical 

atmospheric  radiometric  temperatures  are  in  the  range  of  20°  -100°  K .  Although  typically 
more  energy  is  emitted  at  IR  than  at  MMW  by  an  object,  MMW  images  tend  to  have  a  higher 
contrast  than  IR  images  [13],  The  higher  contrast  can  be  attributed  to  the  fact  that  cloud  cover 
is  comparatively  transparent  to  millimeter  waves  and  the  sky  temperature  can  be  quite  low 
(typically  less  than  100°K).  Thus  an  object  at  300°  K  will  have  an  emissivity  that  can  produce 
a  brighter  signature  resulting  from  the  cold  sky  reflections.  The  effects  of  image  contrast  on 
the  PMMW  super-resolution  performance  are  not  well  understood  at  present.  One  may  note 
however  that  as  the  imaging  frequency  is  increased,  the  acquired  image  tends  to  have  a  higher 
spatial  resolution  for  a  given  antenna  size  at  the  cost  of  reduced  image  contrast.  Thus, 
quantitative  evaluations  of  processing  PMMW  images  of  the  same  scenes  collected  at  35  GHz, 
95  GHz  and  120  GHz  could  help  in  understanding  the  relation  between  the  contrast  in  acquired 
imagery  and  the  spectral  extrapolation  that  could  be  achieved  from  processing  it. 
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2.5  Image  Quality  Measures  and  Assessment  of  Resolution  Improvement 

A  particularly  challenging  question  in  resolution  enhancement  studies  is  devising 
appropriate  metrics  to  assess  the  quality  in  a  processed  image  in  order  to  understand  the 
resolution  improvement  gained  from  implementing  a  particular  algorithm  or  to  compare  the 
relative  performance  of  two  alternate  algorithms.  This  question  attains  particular  significance 
in  the  use  of  iterative  restoration  and  super-resolution  algorithms  since  the  resolution  gains 
achieved  after  a  certain  number  of  iterations  may  not  be  significant  enough  to  justify  the 
investment  in  computational  effort  and  hence  a  criterion  to  stop  the  iterations  may  need  to  be 
established  based  on  some  quantitative  measures  or  numerical  metrics. 

Part  of  the  difficulty  in  assigning  a  precise  metric  for  assessment  of  resolution 
improvements  from  a  given  algorithm  stems  from  the  uncertainty  surrounding  the  question  of 
how  to  measure  resolution  in  a  given  image.  Evidently,  several  definitions  are  possible 
depending  on  what  type  of  information  is  sought.  For  example,  if  separation  of  point  sources 
in  the  scene  being  imaged  is  of  interest,  one  may  attempt  to  use  the  standard  measure  given  by 
the  Rayleigh  criterion  relating  the  angular  resolution  6  to  the  wavelength  of  radiation  X  and 
the  aperture  diameter  D 

X 

0  =  1.22—  . 

This  expression  results  from  the  assumption  that  two  points  on  the  object  or  scene  are  resolved 
in  the  image  produced  if  their  angular  separation  is  such  that  each  point  is  located  at  the  first 
minimum  of  the  diffraction  pattern  of  the  other  [3].  However,  if  the  question  of  interest  is 
sharpening  of  the  edges  and  shapes  in  the  image  being  processed  such  that  specific  objects  can 
be  more  easily  recognized,  the  above  measure  may  not  be  the  right  one  to  use  and  a  different 
metric  that  appropriately  quantifies  the  edge  sharpening  achieved  would  be  needed. 
Furthermore,  for  images  of  scenes  consisting  of  complex  extended  objects,  the  Rayleigh 
measure  which  is  based  on  the  resolution  of  point  sources  in  the  object  is  clearly  not 
appropriate.  In  these  cases,  it  is  difficult  to  derive  a  satisfactory  quantitative  measure  on  which 
to  base  the  decision  that  objects  in  the  scene  are  resolved  and  the  decision  may  need  to  be 
based  on  visual  perception  which  becomes  a  subjective  measure.  Also,  if  the  processed  image 
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is  used  for  the  extraction  of  certain  features  which  are  then  used  for  some  specific  purpose 
(such  as  target  detection,  object  recognition,  or  fusion  of  images),  the  quality  of  features  that 
can  be  extracted  becomes  the  meaningful  criterion.  Added  to  the  difficulty  is  also  the  diversity 
in  the  possible  measurements  since  a  number  of  statistical,  spectral  and  spatial  metrics  can  be 

used  to  assess  a  given  image. 


Since  the  primary  goal  of  image  super-resolution  is  to  restore  the  image  by  extending 
the  frequency  content  beyond  the  cutoff  frequency  of  the  sensor  ®e,  it  is  natural  that  any 
quality  metric  used  to  score  a  super-resolution  algorithm  must  include  the  size  of  the  extension 
achieved.  However,  such  a  measurement  is  possible  when  the  ideal  image  of  the  object  bemg 
imaged  (that  contains  the  entire  spectrum)  is  available  for  comparison.  In  the  more  practical 
cases  where  the  ideal  image  with  which  the  processed  image  can  be  compared  is  not  available, 
one  may  need  to  use  some  indirect  ways  of  quantifying  the  improvement  in  the  size  of  the 
spectrum  achieved  from  super-resolution  processing.  Consequently,  in  the  further  discussion 
we  will  distinguish  between  these  two  cases. 


When  the  ideal  image  is  available  for  comparison,  various  distance  metrics  can  be 
readily  postulated  to  compare  the  images  and  their  spectra.  Straightforward  measures  are  the 
L  -norms  of  the  deviations  in  the  image-domain  or  in  the  frequency-domain  given  by 


a/./)=hks.M' 


(9) 


or 


1 

^  P 


(10) 


where  /  denotes  the  ideal  image  of  size  M  by  N,  f  the  super-resolved  image  of  the  same  size. 


and  F  and  F  denote  the  corresponding  spectra.  A  problem  of  particular  concern  when  using 
distance  measures  of  the  above  types  is  that  elements  with  very  high  or  very  low  magnitudes 
can  unduly  influence  the  value  of  the  metric  computed.  Consequently,  a  measure  of  the 
correlation  between  the  two  images  being  compared  may  be  more  appropriate.  Especially 
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useful  is  a  measure  of  the  similarity  between  the  images  and/or  their  spectra  obtained  from 
comparing  blocks  of  pixels  from  which  a  'Similarity  Map"  may  be  constructed  as  follows: 

1.  Each  of  the  two  images  to  be  compared,  of  size  M  by  N  pixels,  is  subdivided  into  a  set  of 
smaller  images,  each  of  size  K  by  L  (a  K  =  M,  a  s  Integer  and  /?  L  =  N  ,  P  £  Integer  ). 

The  smaller  image  blocks  do  not  overlap  but  completely  cover  the  original  image 

2.  Then,  KL  similarity  values  are  computed  by  evaluating  the  cross-correlation  between  the 
corresponding  image  blocks  in  the  ideal  image  and  in  the  processed  image  which  is  defined 

by 
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where  /;(/)  is  the  i'h  component  of  the  point  that  represents  the  m"’ block  obtained  from 
the  reference  image,  and  /;(/)  is  the  corresponding  block  obtained  from  the  processed 


image. 

3.  The  resulting  KL  similarity  values  are  displayed  using  the  same  metric  that  orders  the  set 
of  smaller  images.  In  order  to  obtain  a  similarity  map  with  as  many  dimensions  as  the 
two  images  being  compared,  an  M  by  matrix  can  be  divided  in  the  same  manner  as  the 
reference  and  the  processed  images  are  divided.  Then,  the  similarity  value  obtained  from 
each  pair  of  smaller  images  can  be  copied  into  all  the  pixels  of  the  corresponding  smaller 

matrix  of  the  similarity  map. 


The  similarity  map  obtained  as  above  can  then  be  used  to  compute  various  metrics  that 
can  yield  different  comparison  scores.  One  particular  example  of  using  such  a  Similarity  Map 
obtained  from  comparing  the  spectra  of  the  ideal  reference  image  and  the  processed  image  m 
order  to  compute  a  “virtual  aperture  diameter”  that  can  effectively  compare  two  alternate 
super-resolution  algorithms  will  be  described  in  a  later  section. 
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When  the  ideal  image  is  not  available  for  comparison,  development  of  appropriate 
metrics  becomes  more  difficult  and  should  necessarily  be  based  on  using  the  original  image 
being  processed  in  the  comparison  process.  In  comparing  statistic-based  restoration 
procedures,  such  as  the  ML  and  MAP  algorithms,  certain  statistical  parameters  can  yield  a 
mechanism  for  useful  comparison.  Some  examples  are  the  likelihood  measure  given  by 


L  (g,h,f)  =  p(g / h,f)  =  Yl 
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and  the  posterior  distribution  given  by 

L  (g,hj,f)  =  p(g/h’f’f)  = 
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In  the  above  expressions,  g  denotes  the  acquired  image  being  processed,  h  denotes  the  sensor 
PSF,  /is  the  processed  image,  and  /  is  the  image  prior.  Alternately,  one  can  use  the  value  of 

the  “residual”  given  by 


(14) 


Yet  another  way  of  comparing  the  restored  image  with  the  original  image  is  to  extract  a  slice 
of  data  from  their  spectra  and  comparing  the  intensity  levels  at  specified  frequency  values. 
More  details  on  such  a  comparison  will  be  given  in  the  next  section  when  evaluating  the 
performance  of  the  ML  super-resolution  algorithm.  It  is  to  be  noted  however  that  such 
evaluations  are  only  of  a  subjective  nature  since  an  arbitrary  determination  of  which  frequency 
values  are  to  be  used  for  comparison  will  have  to  be  made. 
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3.  RESTORATION  AND  SUPER-RESOLUTION  PERFORMANCE  OF  ITERATIVE 
ALGORITHMS  OBTAINED  BY  A  MAXIMUM  LIKELIHOOD  APPROACH 

3.1  Derivation  of  an  Updating  Rule  for  Iterative  Maximization  of  Likelihood 

An  important  class  of  algorithms  that  are  receiving  particular  attention  in  recent  times 
for  their  super-resolution  capabilities  are  those  that  can  be  developed  starting  with  a  statistical 
modeling  of  the  imaging  process.  The  basic  idea  underlying  these  methods  is  to  account  for  the 
statistical  behavior  of  emitted  radiation  at  the  level  of  individual  photon  events  by  constructing 
appropriate  object  radiance  distribution  models  (using  knowledge  of  fluctuation  statistics).  A 

particularly  attractive  approach  is  to  obtain  a  maximum  likelihood  (ML)  estimate  {/(*,>>)}  i.e. 
the  object  intensity  estimate  that  most  likely  have  created  the  measured  data  {gM}  with  the 
PSF  process  [h{z„z2)) ,  which  in  turn  is  developed  by  maximizing  an  appropriately  modeled 
likelihood  function  (or  the  logarithm  of  this  function,  for  simplicity),  i.e.  obtaining  f  that 
solves  the  maximization  problem 


f 


=  argmax 


p{s!f)- 


(15) 


Modeling  the  likelihood  function  is  basically  obtaining  a  goodness-of-fit  (GOF)  quantity  for  the 
measured  data,  since  the  likelihood  function  is  a  statistical  distribution  function  p(g/f) 

obtained  as  a  fit  to  the  relation  between  the  data  {g(x,y)}  and  the  object  {/(x,y)}.  The 
success  of  image  restoration  in  a  given  application  depends  on  how  good  the  assumed 
conditional  probability  function  fits  the  input/output  characteristics  of  the  imaging  system. 

Selection  of  an  appropriate  distribution  function  for  modeling  the  likelihood  p(g/f) 
should  also  be  based  on  the  simplicity  of  the  resulting  algorithm  for  practical  implementations. 
A  particularly  simple  iteration  rule  can  be  developed  by  modeling  p(g/f)  as  a  Poisson 
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distribution.  Furthermore,  by  assuming  that  the  individual  pixels  are  statistically  independent, 
one  can  obtain  a  model  for  p(g/f)  as 


p(s/f)  =  n 
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For  maximizing  the  likelihood  one  may  differentiate  p(g/f)  with  respect  to  /  and  solve  the 
equation  dp  /  df  =  0.  However,  considerable  simplification  is  obtained  by  attempting  to 

maximize  the  natural  logarithm  of  the  likelihood  function  l(g/ f)  =  \np(g/ /) .  Hence, 
computing 
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and  setting  it  to  zero,  one  obtains  the  extremum  condition 
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(18) 
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Equation  (18)  needs  to  be  solved  for  /  with  the  given  g  and  h  .  Due  to  the  complexity 
of  the  equation,  one  attempts  to  obtain  an  iterative  solution  in  the  form  given  by 


(19) 


where  F(»)  is  the  updating  function. 


Multiplying  both  sides  of  Equation  (18)  by  f(x,y)  and  rearranging  terms  one  obtains 
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Now,  using  Picard’s  iteration  it  is  possible  to  write  the  expression  for  the  updating  rule  in  the 
form 


fn+'(x,y)  =  a  f”(x,y) 
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where 


a  = 
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and  the  symbol  *  denotes  correlation. 
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If  the  PSF  of  the  imaging  system  is  symmetric  and  is  normalized  such  that  the  sum  of 
its  values  is  one,  i.e. 


Yjh(x'  -x,y'  -y)  =  1 

h{x' -x,y -y)  =  h(x-x',y-y') 


(23) 


the  above  iteration  can  be  rewritten  in  the  final  form 
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which  yields  a  systematic  procedure  for  constructing  the  object  estimate  fn(x,y)  from  the 
given  image  g(x,y)  and  the  PSF  function  h(x,y) .  For  commencing  the  iteration,  one  can  use 
the  starting  estimate  /°(x,.y)  =  g(*,.y)  . 

3.2  Evaluation  of  Restoration  and  Super-resolution  Performance 

In  this  section  we  shall  present  the  results  from  a  few  simple  experiments  of  restoring 
blurred  objects  using  the  iterative  rule  described  by  Equation  (24).  Since  our  interest  in  this 
project  is  not  merely  in  the  restoration  within  passband  but  also  extrapolation  of  the 
frequencies  the  changes  to  the  image  spectrum  due  to  the  application  of  the  algorithm  will  be 
given  a  special  emphasis . 

Experiment  1: 

Fig.  2a  shows  a  one-dimensional  object  characterized  by  several  edges.  Fig.  2b  shows 
the  image  formed  when  blurred  with  a  sensor  with  a  cutoff  frequency  21.  Fig.  2c  shows  the 
reconstructed  image  after  90  iterations  of  the  ML  algorithm  described  by  Equation  (24).  While 
the  edge  reconstruction  and  the  restoration  of  the  original  object  are  clearly  seen,  the  spectrum 
extrapolation  (and  hence  super-resolution)  can  be  more  clearly  seen  by  examining  the 
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frequency  spectra  of  the  signals  before  and  after  processing.  Fig.  2d  shows  the  spectrum  of  the 
original  object,  Fig.  2e  shows  the  spectrum  of  the  image  formed  (note  the  cutoff  frequency  21 
of  the  sensor)  and  Fig.  2f  shows  the  spectrum  of  the  reconstructed  image  after  90  iterations  of 
the  ML  algorithm.  It  should  be  noted  that  in  these  figures  only  the  portion  of  the  spectrum  in 
the  high  frequency  range  is  plotted  to  an  expanded  scale  since  the  signal  has  low  frequency 
components  with  relatively  large  magnitudes  that  prevent  the  high  frequency  portion  of  the 
spectrum  from  being  displayed  effectively  on  the  same  graph.  The  extrapolation  of  frequency 
components  beyond  the  cutoff  of  21  clearly  demonstrates  that  this  algorithm  is  super-resolvmg. 


Fig.  2a-2f.  Results  of  processing  one-dimensional  signal  in  Experiment  1. 
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Experiment  2: 

For  illustrating  the  convergence  behavior  of  the  iterations  and  the  performance  of  the 
ML  algorithm  given  by  Equation  (24)  in  processing  a  two-dimensional  image,  a  controlled 
experiment  was  conducted  on  a  simulated  object  comprising  of  a  series  of  concentric  disks  with 
the  background  (dark)  at  intensity  value  zero  and  the  disks  (bright)  at  intensity  value  one, 
shown  in  Fig.  3a.  For  simulating  the  blurring  caused  by  a  diffraction  limited  imaging  sensor, 
this  object  was  convolved  with  the  PSF  of  a  low-pass  filter  that  simulates  a  sensor  with  a 
circular  aperture  of  diameter  16  pixels.  The  blurred  image  is  shown  in  Fig.  4a.  The  effects  of 
this  blurring  operation  can  be  seen  by  comparing  the  spectra  shown  in  Figs.  3b  and  4b,  which 
display  the  extent  of  frequencies  contained  in  the  original  object  of  Fig.  3a  and  the  blurred 
image  in  Fig.  4a  respectively.  To  facilitate  a  digital  restoration  processing,  the  blurred  image 
in  Fig.  4a  is  represented  on  a  512x512  grid  of  samples,  which  evidently  constitutes  an 
oversampling  of  this  image  (since  no  frequency  components  beyond  the  cutoff  frequency  of  16 
are  present). 


Figs.  3a  and  3b.  Object  represented  on  a  512  x  512  grid  and  its  spectrum 


Fig.  5a  shows  the  restored  image  obtained  after  50  iterations  of  processing  the  blurred 
image  with  the  ML  algorithm  given  by  Equation  (24).  The  removal  of  the  blurring  effects  are 
clearly  evident.  However,  of  particular  interest  also  is  the  super-resolution  performance  of  the 
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algorithm,  viz.  the  extrapolation  of  the  image  spectrum.  The  spectrum  of  this  restored  image  is 
shown  in  Fig.  5b  which  clearly  depicts  the  enlargement  of  the  region  that  contains  the  spectral 


Figs.  4a  and  4b.  Blurred  image  and  its  spectrum 


Figs.  5a  and  5b.  ML  restored  image  and  its  spectrum 


components,  thus  verifying  the  super-resolution  capabilities  of  the  algorithm.  For  obtaining  an 
understanding  of  the  convergence  behavior  of  the  algorithm,  the  processing  was  continued  for 
1000  iterations  and  the  normalized  restoration  error  given  by  the  L, -norm  of  the  deviation 

f(x,y)-f(x,y )  was  examined.  Fig.  6  depicts  the  behavior  of  the  restoration  error  as  the 
number  of  iterations  is  increased. 
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Iterations 


Fig.  6.  Convergence  of  restoration  error 

A  quantitative  evaluation  of  the  restoration  gain  can  also  be  made  in  this  case  by 
computing  the  “virtual  aperture”  that  likely  will  acquire  the  image  that  resulted  from  the  signal 
processing.  To  enable  this  computation  one  may  obtain  the  Cross-correlation  Similarity  Maps 
comparing  the  spectra  of  the  unprocessed  and  processed  images  with  the  spectrum  of  the 
original  ideal  object  as  outlined  in  Section  2.5.  Figs.  7a  and  7b  respectively  show  these 
similarity  maps  in  the  Fourier-domain  obtained  by  comparing  the  blurred  image  and  the  ML 
super-resolved  image  after  100  iterations  with  the  ideal  object.  In  these  images,  a  value  of  zero 
is  shown  with  black  color  and  the  images  are  scaled  such  that  the  maximum  value  is  assigned 
as  pure  white.  One  particular  advantage  of  these  maps  is  that  they  allow  to  precisely  point  out 
those  parts  of  the  spectrum  that  have  been  restored  (white  spots)  and  also  the  parts  where  the 
processing  has  deteriorated  the  spectrum  (black  spots). 
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Fig.  7a  and  7b.  Cross-correlation  similarity  maps  in  Fourier-domain 


For  an  imaging  system  with  a  circular  aperture,  the  frequency  components  of  any 
acquired  image  should  reflect  this  fact  by  having  all  information  within  a  circle  of  radius  equal 
to  the  cutoff  frequency.  For  the  blurred  image  shown  in  Fig.  4,  this  value  is  straightforward: 
the  cutoff  frequency  is  twice  the  radius  of  the  aperture  function  used  to  blur  the  original  object 
shown  in  Fig.  3.  For  the  super-resolved  image  one  can  define  a  corresponding  parameter  as  the 
cut-off  frequency  imposed  by  a  “virtual  camera”  that  is  needed  to  acquire  an  image  equivalent 
to  this  one,  which  can  in  turn  be  quantified  by  computing  the  Virtual  Aperture  Diameter 
(VAD).  One  way  of  measuring  the  VAD  will  be  described  in  the  following.  In  order  to 
evaluate  how  the  extent  of  similarity  is  varying  along  the  frequency-axis,  an  annular  ring  with 
width  of  a  certain  chosen  value  (say  11  pixels)  is  centered  in  the  similarity  map.  Pixels  lying 
within  this  ring  are  weighted  by  their  energies  (square  of  the  intensity  values)  and  then 
averaged.  Now,  as  the  radius  of  the  ring  is  progressively  increased  from  the  center  of  the 
similarity  map,  the  averages  computed  will  yield  a  plot  that  describes  how  the  average 
similarity  is  varying  as  a  function  of  the  frequency.  Such  plots  are  shown  in  Fig.  8  where  the 
bottom  curve  shows  the  variation  of  the  average  similarity  as  a  function  of  frequency  for  the 
blurred  image  while  the  top  curve  shows  the  corresponding  variation  for  the  super-resolved 
image. 
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Average  Similarity  vs.  Frequency 


Fig  8.  variation  of  average  similarity  vs.  frequency 

The  extended  cutoff  frequency  and  the  VAD  can  now  be  readily  calculated.  Starting 
with  the  cutoff  frequency  of  16  for  the  blurring  employed,  one  observes  that  the  average 
similarity  value  at  this  frequency  is  0.5475.  For  the  super-resolved  image,  the  highest 
frequency  at  which  an  average  similarity  value  no  less  than  0.5475  can  be  determined  as  115, 
which  gives  the  value  of  VAD  in  this  case.  It  may  be  noted  that  an  improvement  by  a  factor  of 
7.1875  is  achieved  with  respect  to  the  original  sensor  aperture  size  of  16  pixels  due  to  the  ML 
processing. 

Experiment  3; 

To  evaluate  the  super-resolution  performance  of  the  algorithm  on  a  real  degraded  image 
an  experiment  was  conducted  by  processing  a  256x256  image  from  our  database.  Fig.  9a 
shows  the  original  image  (“Lena”)  used  in  this  experiment.  Fig.  9b  shows  the  blurred  image 
obtained  by  convolution  with  the  PSF  of  a  sensor  with  cutoff  coc  =  63  (which  is  approximately 
one-half  of  the  folding  frequency  and  is  typical  of  practical  imaging  operations).  The 
restoration  performance  of  ML  algorithm  as  the  number  of  iterations  is  gradually  increased  is 
shown  in  the  next  set  of  figures  (Figs.  9c  and  9d  where  the  restored  images  after  10  and  100 
iterations  are  shown).  The  resolution  enhancement  with  only  10  iterations  of  the  algorithm  is 
clearly  visible.  The  algorithm  was  stopped  at  the  end  of  100  iterations  since  the  resolution  is 
comparable  to  that  of  the  original  (unblurred)  image.  Figures  9e  -  9h  show  the  power  spectra 
of  various  signals;  the  frequency  extrapolation  performed  by  the  ML  algorithm  to  achieve 
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super-resolution  is  clearly  noticeable  by  comparing  the  four  comers  in  Figs.  9g  and  9h  which 
show  the  power  spectra  of  the  blurred  image  and  the  reconstructed  image  after  100  iterations  of 
the  algorithm. 


Fig.  9a.  Original  object  Fig.  9b.  Blurred  image 


Fig.  9c.  Processed  image  after 
10  ML  iterations 


Fig.  9d.  Processed  image  after 
100  ML  iterations 


Fig.  9e.  Spectrum  of  object  Fig.  9f.  Spectrum  of  blurred  image 
Fig.  9.  Results  of  processing  two-dimensional  image  in  Experiment  3. 
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Fig.  9g.  Spectrum  of  processed  image  Fig.  9h.  Spectrum  of  processed  image 

after  10  Iterations  after  1 00  iterations 

Fig.  9  (continued).  Results  of  processing  two-dimensional  image  in  Experiment  3. 


3.3  Analytical  Properties  of  the  ML  Iteration  Algorithm 

When  iterative  restoration  algorithms  are  to  be  considered  for  practical 
implementations,  a  very  important  issue  is  the  convergence  of  iterations.  Unlike  in  the  case  of 
other  iterative  algorithms  for  image  restoration  and  super-resolution,  some  convergence  results 
for  the  ML  algorithm  given  by  Equation  (24)  can  be  analytically  proven.  For  instance, 

computing  the  second  derivative  of  the  log-likelihood  function  L,{g / /) , 
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The  quantity  on  the  right  hand  side  is  clearly  negative,  since  the  image  intensity  values 
g(x’,y ’)  are  always  non-negative.  This  result  implies  that  L^g/f^j  and  p{g/f )  are 
monotonically  increasing  functions  with  respect  to  f(x,y) ,  i.e.  the  likelihood  improves  with 
each  successive  iteration. 

Although  convergence  of  the  ML  iterations  can  be  proven  as  above,  the  rate  of 
convergence  depends  on  a  number  of  factors,  most  notably  on  the  accuracy  with  which  the  PSF 
of  the  imaging  process  h(x,y)  is  known  and  the  Signal-to-Noise  Ratio  (SNR)  of  the  image 
being  processed.  It  is  usually  observed  in  practice  that  after  a  steep  convergence  during  the 
first  few  iterations,  the  later  iterations  tend  to  get  much  slower  in  the  sense  of  diminishing 
restoration  quality  improvement  following  each  iteration.  Furthermore,  if  significant  amounts 
of  noise  are  present  or  if  significant  inaccuracies  in  the  assumed  PSF  exist,  the  restoration 
artifacts  grow  progressively  with  each  iteration  resulting  in  poor  convergence  (or  even 
divergence)  in  some  cases.  Consequently,  one  will  invariably  require  the  termination  of  the 
algorithm  after  a  certain  number  of  iteration  steps  or  the  implementation  of  appropriate 
modifications  such  as  PSF  correction  (leading  to  Blind  ML  Deconvolution)  and  post-filtering 
steps  as  will  be  described  in  a  later  section. 


The  ML  iterative  algorithm  described  by  Equation  (24)  was  previously  studied  for  its 
passband  restoration  properties  and  attracted  considerable  notice  by  the  radio-astronomy  and 
medical  imaging  communities  [21-23],  Very  recently,  however,  the  super-resolution 
capabilities  of  this  algorithm  (i.e.  bandwidth  extrapolation  properties)  are  being  increasingly 
studied,  specifically  for  processing  diffraction  limited  imagery  acquired  in  military  surveillance 
and  guidance  applications  [24-26],  The  super-resolution  capabilities  stem  from  the  nonlinear 
processing  involved  in  each  iteration  step  of  the  algorithm.  Furthermore,  the  required  updating 
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inherently  implements  the  non-negativity  of  the  object  estimate,  which  is  an  a  priori 
information  about  the  object  that  needs  to  be  imposed  as  a  constraint  in  other  iterative 

restoration  algorithms. 


The  ML  iterations  specified  by  Equation  (24)  result  in  a  nonlinear  dynamical  scheme  of 
the  form 


gU)  } 


(26) 


where  «(/,(/))  =  0)J-  The  conver«ence  of  iterationS  Ca" 

hence  be  examined  in  terms  of  the  equilibrium  points  of  this  dynamical  system,  or  equivalently 
of  the  “fixed  points”  fk(J)  that  satisfy 


o(/*  (/>)  =  fkU)-  (27) 

It  is  easy  to  see  that  this  condition  is  attained  at  the  k- th  iteration  when  the  restored  image 
satisfies  the  equality 


g(J)  =  h(J) . 

since  the  quantity  inside  the  square  brackets  in  the  description  of  O (/*(;'))  becomes  1  (when 
the  PSF  is  symmetric  and  normalized,  which  in  turn  implies  that  1  ®  h(j)  =  1).  Let  the  image 

estimate  vector  {/’O')}  denote  this  converged  value.  Thus 

gU)  =  f'U)®h(j)  (28> 


specifies  the  fixed  points  {/*(/)}  of  the  ML  restoration  algorithm  given  by  Equation  (24). 
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A  careful  examination  of  the  condition  specified  by  Equation  (28)  reveals  several 
interesting  facts.  First  of  all  observe  that  Equation  (28)  corresponds  to  the  condition  in  the 
frequency  domain 


G(jco)  =  F\jco)H(jco)  (29) 

and  hence  is  satisfied  by  any  estimate  {/‘O')}  that  restores  the  passband  spectrum.  Due  to  the 
band-limiting  operation  of  the  convolution  with  the  PSF  vector  {/*(/)} ,  the  image  estimate 
{/*(;)}  that  satisfies  Equation  (28)  is  not  unique.  Since  equality  outside  the  passband  of  the 
sensor  is  trivially  satisfied  (i.e.,  both  the  left  and  the  right  sides  of  Equation  (29)  become 
zero),  there  exist  an  infinite  number  of  vectors  {/*(/)}  that  make  Equation  (28)  hold.  These 
vectors  are  however  related  in  the  sense  that  each  of  them  has  a  specific  structure  that  permits 
it  to  be  written  as  the  sum  of  a  fixed  vector  {//*(/)}  that  matches  the  object  spectrum 
F(jco)  identically  below  the  diffraction  limit  (i.e.,  for  <x>  <coc)  and  another  vector 
{/I  O')}  with  arbitrary  values  but  with  the  spectrum  below  the  diffraction  limit  identically 

zero.  In  other  words,  any  estimate  {/‘O')}  that  can  be  written  as 

/*o')=/;o)+/;Ao)  (30) 

where  f’pb(j)  and  f*b(j)  satisfy  the  frequency-domain  conditions: 

(i)  H(jco)F*b(jco)  =  G(jco) 

(ii)  F*b(jco)  =  0  for  co  <  coc ,  (31) 

serves  as  a  fixed  point  of  the  ML  algorithm. 
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Bayesian  estimation  methods  in  general,  and  ML  algorithm  in  particular,  that  iteratively 
attempt  to  maximize  the  likelihood  of  the  estimate  [fk  (/)} ,  which  gives  rise  to  the  image 
vector  {g(/)}upon  convolution  with  the  PSF  vector  {/z(/)} .  attempt  to  enforce  the  passband 

restoration  condition  given  by  (i)  above.  Hence,  when  an  estimate  {/*(./)}  that  satisfies  this 

condition  is  obtained  at  the  k-th  iteration,  the  algorithm  ceases  to  do  any  further  work. 
Consequently,  convergence  of  the  algorithm  is  to  be  interpreted  only  in  terms  of  restoration  of 
the  passband  spectrum  of  the  image.  In  other  words,  these  algorithms  principally  strive  to 
restore  the  passband  as  accurately  as  possible  with  the  following  resultant  effects: 

(i)  Since  there  is  no  special  incentive  for  obtaining  the  part  {//*(./)}  that  matches  the  object 
spectrum  outside  the  diffraction  limit  specified  by  coc ,  this  vector  obtained  at  the  end  of 
several  iterations  (when  convergence  has  occurred)  could  have  elements  whose  contribution 
to  the  frequency  spectrum  is  much  smaller  than  the  contribution  of  the  other  part  |/pA0')|  • 

In  other  words,  the  “energy  in  the  frequency  extension”  could  be  very  small  (or  even 
negligible). 

(ii)  When  passband  restoration  is  completed  (i.e.,  when  convergence  has  occurred),  no  further 
changes  in  the  estimate  {/*(/)}  are  likely,  since  the  likelihood  function  will  have  attained 
its  maximum  value. 

(iii)  Any  tendencies  of  the  later  iterations  of  the  algorithm  to  degrade  the  passband  restoration 
performance  are  resisted  by  the  algorithm,  since  the  algorithm  inherently  is  a  convergent 
one  that  converges  to  one  of  the  several  fixed  points  that  provide  a  perfect  match  of  the 
passband  spectrum. 

The  analysis  given  above  on  the  convergence  behavior  of  the  ML  iterations  is  very 
helpful  in  obtaining  an  insight  into  the  nature  of  the  multiple  fixed  points  of  the  algorithm.  Due 
to  the  infinite  number  of  fixed  points  and  the  corresponding  trajectories  that  describe  the 
convergence  of  the  initial  estimates  to  these  equilibrium  conditions,  obtaining  any  quantitative 
estimates  of  the  convergence  rates  is  rather  difficult.  The  problem  can  be  somewhat  simplified 
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by  examining  the  dynamics  of  the  “iteration  error’,  or  change  in  the  estimate  during  successive 
iteration  steps,  defined  by 

ek(j)  =  fkU)-fk-iU)  ■  W 

The  behavior  of  this  error  is  governed  by  the  nonlinear  dynamical  process 

e„.,  u)  =  O')  -  /» 0)  =  ®(/.  o))  -  /,  0)  <33> 

which  can  be  re-written  as 


«»„o)-,K«.w)  (34) 

by  expressing  /*(;)  in  terms  of  ek(j) .  Equation  (34)  represents  a  nonlinear  dynamical 

A  , 

system  whose  trajectory  behavior  starting  from  an  initial  state  e0(J)  =  f0(j)  -  S(J)  1S  °f 
interest.  The  equilibrium  points  of  this  “error  system”  are  at  the  locations  where  'J/(e*(./))  — 
0,  which  implies  o(fk(j))-fkU)  =  0,  and  hence  ek+](j)  =  0.  Thus  the  multiple  fixed 
points  of  the  ML  algorithm  {/’(./)}  (  on  the  vV2  -dimensional  space  for  the  estimate  /  )  are 

mapped  into  the  unique  equilibrium  point  {e(J)  =  o} ,  which  is  the  origin  of  the  N2- 
dimensional  space  for  the  iteration  error.  Hence  the  convergence  rate  for  the  restoration  can  be 
studied  in  terms  of  the  trajectory  behavior  of  the  “error  system”  given  by  Equation  (34)  near 
this  equilibrium  point. 

For  small  values  of  ek(j)  (<'•<?.,  near  convergence  to  this  equilibrium),  T^C/))  can  be 
linearized  by  a  Taylor  series  in  the  form 

T^O'))  =  Mek(j)  +  [  Higher  Order  Terms  ]  (35) 
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where  M  is  the  N2  x  N2  Jacobian  matrix  M  =  /  dek.  Ignoring  the  Higher  Order  Terms  in 

the  expansion,  the  iteration  error  dynamics  can  now  be  approximated  by  the  linear  system 

<36> 

Solving  this  system  of  equations  yields  the  trajectory  behavior  on  the  At2 -dimensional  error 
space  which  can  be  described  by 

ek(j)  =  Mke0(j)  (37) 

where  Mk  is  the  state  transition  matrix  of  the  system  and  e0(j)  specifies  the  starting  error. 

From  the  convergent  nature  of  the  ML  algorithm  it  follows  that  all  eigenvalues  of  M  are 
inside  the  unit  circle,  i.e.,  \x,{M)\  <  1  where  \{M)  ,  /  =  1,2,  N2 ,  denote  the  eigenvalues 
of  M.  Consequently,  the  rate  of  decay  of  the  error  ek(j)  is  larger  during  the  first  few  iterations 
of  the  algorithm  and  progressively  flattens  as  the  number  of  iterations  grow,  a  fact  that  is 
confirmed  by  simulation  experiments  as  well  (see  Fig.  6). 

3.4  Determination  of  Sensor  Point  Spread  Function 

It  is  evident  from  the  updating  rule  given  by  Equation  (24)  that  a  knowledge  of  h{x,y ) , 
the  sensor  PSF,  is  essential  for  its  implementation.  Indeed,  for  any  given  sensor,  the  accuracy 
with  which  h(x,y )  can  be  modeled  determines  the  performance  of  any  deconvolution 
algorithm  used  to  reverse  the  low-pass  filtering  effects  of  the  sensor  and  achieve  a  desired 
quality  of  restoration. 

If  the  sensor  is  fully  characterized  and  the  imaging  conditions  are  fully  known,  one  may 
attempt  to  model  the  PSF  exactly  and  utilize  it  for  restoration.  Imaging  systems  that  use 
incoherent  illumination  obey  the  intensity  convolution  integral,  which  in  the  frequency  domain 
takes  the  form 
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g(cox  ,  coy )  =  h[o)x  ,  coy  )f(cox  ,  coy ) 


(38) 


where 


H\ 


(o)x,ay)  = 


\\p(co’x,co-y)p‘{cox  -o)x,co-y-coy)dorxdco’y 


CD'x,CO'v 


If  | P(cox,ory) 


dcoxdcoy 


(39) 


In  Eq.  (39),  p(cox,o)y)  is  the  pupil  function  defined  by  the  geometry  of  the  sensor.  This 

equation  states  that  H[cox,coy ) ,  also  called  the  Optical  Transfer  Function  (OTF)  of  the  imaging 
system,  is  defined  by  the  autocorrelation  of  its  pupil  function. 


In  the  case  of  a  circular  pupil,  whose  pupil  function  is: 


CO  <  CO  c 
CO  >  CO  c 


(40) 


where  co  =  Jcox  +  <oy  and  coc  is  the  sensor  cutoff  frequency,  Equation  (39)  can  be  simplified 
into 


co„ 


arcco 


1° 


CO 


1- 


\COj 


CO  <  CO  c 
CO  >  CO  c 


The  cutoff  frequency  coc  can  be  related  to  the  sensor  parameters  by  the  relation 


(41) 


co, .  = 


2  TlJDp 

If 


(42) 
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where  Dp  is  the  diameter  of  the  circular  pupil,  A  is  the  wavelength  of  the  incoming 
radiation,  and  /  is  the  focal  length.  It  may  be  noted  that  for  a  sensor  with  f/1  optics,  coc 

2n 

can  be  determined  simply  from  coc  =  —— .  Once  the  OTF  is  determined  from  Equation 

A 

(41),  the  PSF  can  be  readily  computed  by  taking  its  inverse  Fourier  transform. 

In  practice,  most  incoherent  imaging  systems  sample  the  focal  plane  image  on  a 
discrete  grid  of  samples.  The  sampling  in  the  image  plane  can  be  achieved  either  with  a  fixed 
array  (fully  staring  array)  or  by  scanning,  either  mechanically  or  by  electronic  means,  of  a 
small  number  of  detectors  across  the  image.  The  expression  for<uc ,  given  by  Equation  (42),  is 
useful  for  determining  the  spacing  between  the  detectors  and/or  for  obtaining  a  scan  rate  for 
collection  of  samples. 

While  the  expressions  in  Equations  (41)  and  (42)  help  determine  the  PSF  from  using  the 
sensor  parameters,  the  PSF  can  also  be  experimentally  determined  for  a  given  sensor  by 
attempting  to  image  a  point  source  or  an  edge  source.  A  commonly  used  approximation  to  an 
ideal  point  source  is  to  employ  a  Gunn  diode  oscillator  (GDO)  source  whose  separation  from 
the  sensor  aperture  can  be  adjusted  to  appear  as  a  point  source  [27].  The  distribution  at  the 
focal  plane  can  then  be  used  to  model  the  PSF  of  the  sensor.  Even  in  cases  where  a  complete 
characterization  of  the  sensor  and  the  imaging  conditions  are  not  available,  one  may  conduct  an 
analysis  of  the  acquired  image  data  to  approximately  identify  the  cutoff  frequency  coc  [28]. 
Starting  with  an  intensity  profile  corresponding  to  an  edge  of  the  object  in  the  acquired  image, 
an  OTF  can  be  created  as  a  low  pass  filter  function  with  an  adjustable  cutoff  frequency  whose 
convolution  with  an  input  pulse  object  yields  an  outcome  which  matches  the  edge  profile 
selected.  While  this  process  gives  only  a  rough  estimate  of  the  OTF,  by  an  iterative  adjustment 
of  this  estimate  to  implement  a  blind  deconvolution  scheme  [24] ,  a  more  accurate  OTF  can  be 
developed.  Such  a  procedure  is  of  considerable  practical  usefulness  since  even  if  the  sensor 
parameters  are  accurately  known,  during  the  operation  several  external  factors  such  as 
vibration  of  the  sensor  platform,  random  atmospheric  turbulence,  blur  due  to  target  motion  and 
blur  due  to  out-of-focus  imaging  can  change  the  overall  PSF  of  the  imaging  process.  In  this 
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case,  the  PSF  determined  from  using  the  known  sensor  parameter  values  can  be  used  as  the 
initial  estimate  that  can  be  iteratively  refined  through  blind  deconvolution  adjustment  steps  as 
outlined  in  the  next  section. 

3.5  Analysis  of  Processor  Requirements  for  Implementation  of  ML  Algorithm 

As  noted  earlier,  iterative  approaches  to  image  restoration  possess  considerable  advantages 
over  non-iterative  restoration  procedures  which  attempt  to  implement  an  inversion  of  the  image 
formation  process  described  by  Equation  (1)  in  one  step.  Despite  these  benefits,  there  are 
several  claims  in  the  literature  [29]  that  iterative  algorithms  demand  excessive  computational 
resources  not  conducive  for  practical  implementations  and  one  should  settle  for  a  noniterative 
implementation  even  at  the  cost  of  inferior  restoration  and/or  super-resolution  performance. 
The  aim  of  the  present  analysis  is  to  provide  some  quantitative  data  that  demonstrate  viable 
implementations  of  the  iterative  ML  algorithm  given  by  Equation  (24)  with  available 
microprocessors.  In  the  comparison  of  iterative  and  non-iterative  algorithms  for  super¬ 
resolution,  it  should  be  kept  in  mind  that  iterative  approaches  provide  the  flexibility  for 
applying  the  constraints  that  are  fundamental  to  achieving  the  required  spectral  extrapolation  in 
a  distributed  fashion  as  the  solution  progresses,  and  hence  the  computations  at  each  iteration 
are  generally  far  less  intensive  than  the  single-step  computation  of  non-iterative  approaches 

[28]. 


For  evaluating  the  maximum  processing  time  each  iteration  of  the  algorithm  may 
demand,  an  examination  of  the  mathematical  operations  performed  in  Equation  (24)  indicates 
that  there  are  two  2-dimensional  (2-D)  fast  Fourier  transforms  (FFTs),  two  2-D  Inverse  fast 
Fourier  transforms  (IFFTs),  three  multiplications  (point-by-point)  and  one  division  (point-by- 
point)  performed.  An  approximate  upper  bound  on  the  total  computational  load  can  hence  be 
projected  as  a  requirement  for  six  2-D  FFTs  in  each  ML  iteration  (by  bounding  the  two 
multiplications,  one  division  and  any  other  needed  operations  (for  thresholding,  scaling, 
formatting,  etc.)  by  a  rather  generous  limit  of  two  2-D  FFTs). 
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For  estimating  the  processing  time,  and  the  processing  capacity  in  image  frames/sec,  one 
may  use  a  typical  illustrative  microprocessor  implementation  based  on  TMS320C6x  DSP  chip 
for  which  data  is  readily  available  [30].  For  a  NxN  image,  the  total  number  of  cycles  for 
implementing  a  2-D  FFT  is  given  by 

NC  =  2N  (log4W)x^l0x^  +  33j  +  7  +  -j  (43) 

and  hence  an  upper  bound  on  the  number  of  cycles  for  implementing  two  ML  iterations  can  be 
obtained  as 

N2ML<  24N  (log4  A^lOx^  +  33^  +  7  +  j  (44) 

The  motivation  for  using  two  ML  iterations  in  the  above  computation  comes  from  the 
observation  that  the  convergence  of  the  restoration  error  in  statistical  restoration  algorithms  in 
general,  and  ML  restoration  in  particular,  is  rather  very  steep  during  the  first  few  iterations.  In 
fact,  in  most  practical  implementations  where  the  goal  of  processing  is  super-resolution,  one 
would  notice  a  major  amount  of  spectral  extrapolation  (and  corresponding  resolution 
improvements)  occurring  during  the  first  one  or  two  iterations  with  succeeding  iterations 
adding  progressively  lesser  amounts  of  spectral  components,  Consequently,  in  applications 
where  the  processing  time  is  of  critical  importance,  one  may  stop  the  iterations  on  an  image 
frame  after  the  first  two  and  move  to  the  next  frame. 

Fig.  10a  shows  a  PMMW  image  (“Humvee  image”)  of  size  84x64  on  which  processing 
experiments  were  conducted  to  evaluate  the  resolution  enhancements  resulting  at  the  end  of  two 
ML  iterations.  This  image  was  supplied  to  us  by  the  Wright  Laboratory  Armament 
Directorate.  Only  a  minimal  information  about  the  sensor  that  the  image  was  recorded  by  a 
single  detector  radiometer  with  1ft  diameter  aperture  at  95  GHz  was  available.  The  processed 
image  at  the  end  of  two  ML  iterations  is  shown  in  Fig.  10b  where  the  enhanced  edge  structure 
of  the  vehicle  and  the  improved  resolution  for  identifying  the  details  are  clearly  evident.  Since 
a  complete  characterization  of  the  sensor  and  the  imaging  conditions  were  not  available,  an 
approximate  analysis  was  conducted  to  obtain  the  PSF  for  the  above  processing.  Starting  with 
an  intensity  profile  corresponding  to  a  selected  edge  of  the  object  in  the  image,  a  PSF  was 
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created  to  simulate  a  low-pass  filter  whose  convolution  with  an  input  rectangular  pulse  yields 
an  outcome  which  is  similar  to  the  edge  profile. 


Table  1  gives  quantitative  data  for  different  image  sizes  (N)  of  N2ML  as  well  as  the  time 
in  seconds  for  implementing  two  ML  iterations  and  the  corresponding  processing  capacity  in 
number  of  frames  processed  per  second.  It  is  instructive  to  note  that  a  processing  rate  that 
exceeds  50  frames  of  size  128x128  per  second  (a  rate  that  exceeds  video  rates)  is  supported  by 
this  commercially  available  chip  implementation.  With  additional  efforts  at  optimizing  the 


Number  of  Cycles 

^2  ML 

Time  (in  sec)  for  2  ML  iterations 

Processing  Capacity  (Number  of  frames/sec) 

74,262,528 

0.3713 

2.69 

16,975,872 

0.0848 

11.79 

128 

3,915,264 

0.0195 

51.08 

Table  1.  Typical  processing  capacities  for  iterative  ML  super-resolution. 


implementation,  by  limiting  the  processing  to  only  a  smaller  region  of  interest  in  the  image  for 
instance,  further  increases  in  processing  capacity  can  be  achieved.  Thus,  based  on  the  above 
analysis,  we  conclude  that  practical  implementations  of  iterative  ML  super-resolution 
algorithms  of  the  type  discussed  here  are  quite  realistic  even  in  applications  where  processing 
time  is  of  critical  importance  and  hence  one  need  not  settle  for  non-iterative  processing 
algorithms  that  may  not  provide  all  of  the  advantages  (robustness,  quality  of  restoration, 
suppression  of  noise  effects,  etc.)  that  well-tailored  iterative  ML  algorithms  are  capable  of 
providing. 
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3.6  Super-resolution  of  PMMW  Images 

In  this  section  we  shall  outline  the  results  of  a  number  of  processing  experiments  that 
were  conducted  to  assess  the  capability  of  the  ML  algorithm  described  by  Equation  (24)  in  the 
restoration  and  super-resolution  of  PMMW  images.  During  the  course  o  this  project  we 
worked  very  closely  with  a  number  of  research  groups  that  were  developing  state-of-the-art 
PMMW  radiometers.  Consequently,  we  were  able  to  obtain  a  number  of  PMMW  images 
acquired  from  different  sensor  designs  and  collected  under  different  conditions  which  has 
resulted  in  a  large  database  of  PMMW  imagery  data  available  in  our  research  laboratory 
(Information  Processing  &  Decision  Systems  Lab)  at  the  University  of  Arizona.  The  data-sets 
that  we  specifically  used  for  processing  with  the  ML  algorithm  came  from  three  camera 
designs:  (i)  a  single  detector  95  GHz  radiometer  with  1  ft  aperture  built  by  Intelligent  Machine 
Technology  (IMT)  corporation  for  the  Air  Force  Research  Laboratories  (AFRL),  (ii)  a  PMMW 
camera  built  by  TRW  Inc.  that  operates  at  90  GHz  with  a  17  Hz  framerate,  and  (iii)  a  95  GHz 
sensor  built  by  the  Army  Research  Laboratories  (ARL)  with  a  3  ft  parabolic  antenna.  It  may  be 
mentioned  that  while  the  images  from  the  TRW  camera  were  captured  from  an  airborne 
platform  when  the  camera  was  flown  on  a  UH-1N  helicopter  over  Camp  Pendleton  in 
California  and  the  images  were  acquired  from  altitudes  between  500  and  2000  ft  at  a  viewing 
angle  of  45  deg  from  nadir,  the  images  from  the  ARL  and  the  AFRL/IMT  radiometers  were 
acquired  from  a  ground-based  platform.  Furthermore,  these  images  were  collected  at  different 
ranges  and  during  different  times  of  day  thus  providing  a  non-uniform  data-set  for  testing  the 
efficiency  of  the  algorithm. 

Fig.  11a  shows  the  image  of  a  tank  (a950.sdt)  acquired  by  the  TRW  camera  from  an 
altitude  of  800  ft.  The  poor  resolution  in  the  image  is  evident  and  the  features  of  the  object  are 
not  identifiable.  In  Fig. lib  is  shown  the  processed  image  after  12  iterations  of  the  ML 
algorithm,  which  clearly  demonstrates  the  enhancement  of  resolution.  For  the  sake  of 
displaying  all  the  features  of  the  tank  that  were  brought  up  by  super-resolution  processing,  the 
processed  image  is  also  shown  in  Fig.  11c  after  an  additional  step  of  histogram  equalization. 
This  last  step  merely  adjusts  the  image  contrast  and  does  not  lead  to  any  further  resolution 
enhancement,  and  hence  does  not  require  any  significant  computation.  One  can  easily 
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appreciate  the  restoration  of  the  original  image  and  the  benefits  of  iterative  processing  for 
recognition  and  classification  of  the  object. 


Fig.  11a.  Acquired  image  (a950.sdt) 


Fig.  lib.  ML  Processed  Image  ( after  Fig.  11c.  ML  processed  Image  (after 

12  iterations)  12  iterations  and  histogram  equalization ) 


Fig.ll.  Results  of  processing  “Tank  Image"  (a950.sdt) 

In  another  experiment,  we  processed  the  PMMW  image  (cl0869.sdt)  shown  in  Fig. 
12a  which  was  also  acquired  by  the  TRW  camera.  This  is  the  image  of  a  runway  with  some 
buildings  on  the  side  and  was  taken  from  an  altitude  of  2000  ft.  Fig.  12b  shows  the  processed 
image  obtained  after  10  iterations  of  the  ML  algorithm.  The  same  image  with  the  additional 
step  of  histogram  equalization  is  shown  in  Fig.  12c.  Once  again,  the  resolution  improvements 
obtained  from  super-resolution  processing  are  clearly  evident. 
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Fig.  12a.  Acquired  image  (cl0869.sdt) 


Fig.  12b.  ML  Processed  Image  ( after  Fig.  12c.  ML  processed  Image  (after 

10  iterations)  10  iterations  and  histogram  equalization ) 

Fig.  12.  Results  of  processing  “Runway  and  Buildings  Image”  (cl0869.sdt) 

Fig.  13a  shows  the  image  of  a  tank  (tl2modl2.tif)  acquired  by  the  AFRL/IMT 
radiometer.  The  processed  image  after  25  iterations  of  the  ML  algorithm  is  shown  in  Fig.  13b 
and  the  same  image  with  additional  histogram  equalization  is  shown  in  Fig.  13c.  To  display  the 
additional  enhancement  in  resolution,  the  processing  was  continued  for  100  iterations.  Fig.  13d 
shows  the  resulting  image  at  the  end  of  100  iterations  and  the  same  image  with  additional 
histogram  equalization  is  shown  in  Fig.  13e.  The  progressive  improvements  in  the  tank 
features  particularly  at  the  gun  barrel  and  the  wheels  as  well  as  in  the  background  (the  tree 
branches)  are  to  be  noted. 


Fig.  13a.  “Tank”  image  (tl2modl2.tif)  acquired  by  AFRL/IMT  radiometer 


Fig.  13b.  ML  Processed  image  Fig.  13c.  ML  Processed  image 

(after  25  iterations)  ( after  25  iterations  and  histogram  equalization) 


Fig.  13d.  ML  Processed  image  Fig.  13e.  ML  Processed  image 

(after  100  iterations)  (after  100  iterations  and  histogram  equalization ) 

Fig.  13.  Results  of  processing  AFRL  “Tank  Image”  (tl2modl2.tif) 

Another  PMMW  image  of  a  building  “La  Quinta  Inn”  (tl2modl2.tif)  acquired  from  the 
same  AFRL/IMT  camera  is  shown  in  Fig.  14a.  The  processed  image  after  25  iterations  of  the 


ML  algorithm  is  shown  in  Fig.  14b  and  the  same  image  with  additional  histogram  equalization 
is  shown  in  Fig.  14c.  The  image  obtained  by  continuing  the  processing  for  100  iterations  is 
shown  in  Fig.  14d  and  this  image  with  a  further  histogram  equalization  is  shown  in  Fig.  14e. 


Fig.  14a.  “La  Quinta”  image  (tl2modl2.tif)  acquired  by  AFRL/IMT radiometer 


Fig.  14b.  ML  Processed  image  Fig.  14c.  ML  Processed  image 

(after  25  iterations)  (after  25  iterations  and  histogram  equalization) 


Fig.  14d.  ML  Processed  image  Fig.  14e.  ML  Processed  image 

(after  100  iterations)  (after  100  iterations  and  histogram  equalization) 

Fig. 14.  Results  of  processing  AFRL  “  La  Quinta  Inn”  image  (tl2modl2.tif) 
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In  yet  another  experiment  we  processed  two  images  of  a  Bradley  tank  (Brad45a.bmp 
and  Brad90a.bmp)  acquired  by  the  ARL  radiometer  from  two  different  angles  (45  and  90 
degrees)  at  a  range  of  43  meters  and  a  scan  step  size  of  0.1  deg.  These  images  as  well  as  the 
processed  images  after  10  iterations  of  the  algorithm  are  shown  in  Figs.  15  a-b  and  16  a-b.  The 
improvement  in  the  quality  of  the  images  is  clearly  evident.  For  the  sake  of  displaying  all  the 
features  of  the  tank  that  were  brought  up  by  super-resolution  processing,  the  processed  images 
are  also  shown  in  Figs.  15c  and  16c  with  a  reversed  intensity  scale  that  interchanges  the  bright 
and  the  dark  regions  of  the  image. 


Fig.  15b.  ML  Processed  Image  (after  10  iterations ) 
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Fig.  15c.  ML  Processed  Image  (with  intensity  reversal) 

Figs.  15a-c.  Results  of  processing  “Bradley  Tank”  image  ( Brad45a.bmp) 


Fig.  16a.  Acquired  Image  (. Brad90a.bmp ) 


Fig.  16b.  ML  Processed  Image  ( after  10  iterations) 
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Fig.  16c.  ML  Processed  Image  (with  intensity  reversal) 


Figs.  16a-c.  Results  of  processing  “Bradley  Tank”  image  ( Brad90a.bmp) 
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4. 


MODIFIED  VERSIONS  OF  ML  ALGORITHM 


While  the  performance  of  the  ML  algorithm  for  restoring  and  super-resolving  degraded 
images  is  quite  appealing  as  demonstrated  in  the  last  section,  in  practical  applications  there 
could  be  certain  non-ideal  conditions,  such  as  imperfect  knowledge  of  the  PSF  of  sensor  and 
poor  Signal-to-Noise  (SNR)  in  the  image  to  be  processed,  which  may  result  in  a  suboptimal 
performance.  Appropriate  modifications  may  need  to  be  used  to  restore  optimal  performance  in 
these  cases.  In  this  section  we  shall  discuss  a  few  scenarios  where  some  modifications  to  the 
ML  algorithm  could  be  developed  with  an  objective  of  obtaining  improved  performance. 


4.1  Robustness  to  PSF  Uncertainties  and  a  Blind  ML  Algorithm 

It  is  generally  well  known  that  the  quality  of  restoration  depends  critically  on  the  extent 
of  knowledge  of  the  sensor  PSF.  If  the  sensor  is  fully  characterized  and  the  imaging  conditions 
are  fully  known,  one  may  attempt  to  model  the  PSF  exactly  and  utilize  it  for  restoration  as 
discussed  previously.  Unfortunately,  in  a  real  operation  environment  such  as  that  encountered 
by  a  missile  in  flight,  several  external  factors  such  as  the  vibration  of  the  imaging  system, 
random  atmospheric  turbulence,  blur  due  to  target  motion,  blur  due  to  out-of-focus  imaging 
etc.,  can  change  the  overall  PSF  of  the  imaging  system.  These  factors,  together  with  any 
problems  associated  with  poor  calibration  of  the  deployed  sensor,  can  make  any  assumptions  of 
perfect  knowledge  of  PSF  not  entirely  realistic  to  use  in  practice.  Consequently,  a  certain 
degree  of  robustness  to  uncertainties  in  estimating  the  PSF  parameters  is  useful. 


To  appreciate  the  importance  of  this  feature  in  restoration  operations,  it  suffices  to 
consider  a  trivial  restoration  exercise  in  a  simple  noise-free  scenario.  Let  /  =  [  1  o]  be  a 

simple  object  vector  imaged  by  a  sensor  characterized  by  a  PSF  matrix  H  — 


1  0.9 

0.9  1 


.  Then, 


with  n  =  0  in  Equation  (3),  the  image  obtained  is  g  -  Hf  -  [l  0.9]  .  The  closeness  of  the 
two  elements  in  g,  compared  to  those  off  illustrates  the  blurring  caused  by  the  sensor.  If  an 


_j  x 

exact  knowledge  of  H  is  available,  one  may  compute  H  = 


1 

-0.9 


-0.9 

1 


and  restore  the 
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object  by  implementing  f  =  H~]g  which  yields  /  =  [l  of .  However,  if  the  sensor  PSF  is 
modeled  even  with  a  small  uncertainty,  very  drastic  errors  in  object  estimates  could  result.  To 


illustrate,  consider  that  the  actual  PSF  of  the  sensor  is  Haaual  = 


1.1  0.8 

0.8  1.1 


which  results  in  the 


image  g  =  Haclvalf  =  [l.l  0.8]r .  It  may  be  noted  that  the  variation  in  the  values  of  the  image 
elements  is  rather  small.  However,  with  the  sensor  being  modeled  with  matrix  H,  the  use  of 

A  r  -i  f 

H~]  in  image  restoration  yields  the  estimate  /  =  FT'g  =  [ 2  -  lj  ,  which  signifies  a  complete 
breakdown  in  the  restoration  process. 


The  example  cited  above  illustrates  the  significance  of  uncertainties  in  PSF  parameters 
even  in  noise-free  restorations.  The  impact  of  these  uncertainties  attains  a  considerably  greater 
magnitude  when  significant  noise  levels  are  to  be  taken  into  account.  The  interplay  between 
noise  and  PSF  errors  can  be  seen  simply  from  the  convolutional  model  given  by  Equation  (2). 
Let  h(i),  the  estimated  PSF,  contain  an  error  denoted  by  Ah(i) ,  i.e.  the  true  PSF  of  the  sensor 
htme(i)  =  h{i)  -  Ah(i) .  Then, 

dO  =  “  J) + Ah(i  ~  j)]f(j) + «(0 

7=1 


=  'Eh>rue(i-j)f(j)  + 


7=1 


n(i)  +  ^Ah{i-j)f(j) 


7=1 


(45) 


which  indicates  a  change 


in  the  noise  level  from  n(i)  to  n(i)  =  n(i)  +  ^ Ah(i -  j)f(j)  thus 

7=1 


accentuating  its  effects. 


A  super-resolution  algorithm  that  constructs  object  estimates  in  an  iterative  fashion  by 
starting  with  an  approximately  estimated  PSF  of  the  sensor  and  progressively  modifies  the 
shape  of  the  PSF  (or  equivalently,  its  Fourier  transform,  which  is  the  Optical  Transfer 
Function  (OTF))  as  part  of  the  iterative  procedure  can  be  designed  using  updating  rules  similar 
to  that  in  Equation  (24).  This  algorithm  is  developed  using  a  statistical  framework  for  tailoring 
the  implementation  rules  and  aims  at  maximizing  the  likelihood  function  relating  the  image 


62 


data  to  the  object  estimate  at  any  given  stage  of  the  process.  Since  a  joint  estimation  of  both 
object  and  the  sensor  PSF  is  performed,  this  algorithm  performs  a  blind  restoration  of  the  input 
image.  Hence  the  algorithm  is  ideal  to  use  when  the  initial  PSF  estimate  is  known  to  be 
inaccurate  or  an  approximate  PSF  constructed  from  analysis  of  the  image  available  is  all  that 
can  be  had  to  commence  the  restoration  process. 


The  algorithm  iteratively  implements  several  cycles  of  recursive  estimation.  In  each 
cycle  of  implementation,  there  are  two  distinct  steps  that  will  be  executed  which  are  briefly 
outlined  in  the  following: 

Step  1 :  With  the  initial  object  estimate  fe(j)  =  g(j)  and  the  available  PSF  estimate 


ht  (_/) ,  implement  m  iterations  of  object  estimation  by 

sU) 


/„(/)-/.(/) 


®A,0) 


,  k  =  0,l,2,...,(m-l);  j  =  1,2,...,N, 


(46) 


where  k  denotes  the  iteration  count  and  <8>  denotes  discrete  convolution,  as  before. 

Step  2:  With  the  object  estimate  fJJ)  obtained  at  the  end  of  Step  1,  implement  n 
iterations  of  PSF  updating  by 

dd 


K\  O'M/ 0) 


where  /  denotes  the  iteration  count. 


®/.W 


,  l  =  0,l,2,...,(/w-l);  j  =  1,2,...,N, 


(47) 


A  flow-chart  depicting  the  iterative  implementation  of  the  two  steps  described  above  is 
shown  in  Fig.  17.  The  algorithm  can  be  run  iteratively  over  several  cycles  until  a  specified 
maximum  iteration  count  is  reached  or  a  processed  image  with  a  satisfactory  resolution  level  is 
attained. 
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Fig.  17.  Flow  chart  for  implementation  of  Blind  ML  Algorithm. 

The  super-resolution  performance  of  this  algorithm  has  been  tested  through  several 
experiments  conducted  with  both  one-  and  two-dimensional  signals  (including  in  particular 
some  PMMW  images  obtained  with  rather  poor  spatial  resolution).  In  the  following,  we  shall 
demonstrate  the  robustness  of  the  algorithm  to  variations  in  the  starting  estimates  of  PSF 
parameters.  An  insight  into  this  attractive  property  of  robustness  can  be  given  from  the 
underlying  ML  estimation  procedure  employed.  Since,  at  the  most  fundamental  level,  an  ML 
estimation  algorithm  relates  conditional  probabilities  through  statistical  arguments,  it  possesses 
intrinsic  capabilities  to  account  for  statistical  fluctuations  in  the  signals  and  the  imaging 
process. 

In  order  to  conduct  a  controlled  experiment  where  the  errors  in  signal  restoration  (in 
spatial  domain)  and  in  PSF  estimation  (or  equivalently  in  the  estimation  of  OTF,  measured  in 
the  frequency  domain)  could  be  monitored  over  several  cycles  of  algorithm  implementation,  an 
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experiment  was  conducted  starting  with  a  one-dimensional  object  consisting  of  three  pulses  of 
uneven  heights,  shown  in  Fig.  18a.  To  obtain  an  image  to  be  used  as  input  to  the  super¬ 
resolution  algorithm,  a  noisy  blurred  image  was  constructed  by  convolving  the  object  with  a 
sensor  with  cutoff  frequency  coc  =  30  and  a  OTF  with  a  triangular  profile  whose  magnitude 
varies  linearly  from  a  normalized  value  1  at  co  -  0  to  0  at  o)  =  coc=  30  (shown  in  Fig.  18b), 
and  by  adding  Gaussian  noise  to  produce  a  signal  with  SNR  =  14.9391  dB.  The  resulting 
blurred  signal,  shown  in  Fig.  18c,  is  processed  by  the  blind  ML  restoration  algorithm.  Since 
our  present  objective  is  to  demonstrate  the  robustness  of  the  algorithm  to  inaccurate  selections 
of  PSF  parameters  (or  equivalently  OTF),  two  implementations  of  the  algorithm  were  made 
with  two  distinct  OTF  profiles,  one  with  a  cutoff  frequency  28  (hereafter  referred  to  as 
Implementation  1)  and  the  other  with  a  larger  error  made  by  selecting  a  cutoff  frequency  25 
(referred  to  as  Implementation  2).  In  each  implementation,  the  algorithm  was  run  through  10 
cycles  with  each  cycle  comprising  of  five  object  estimation  iterations  (i.e.  m  =  5  in  Step  1) 
followed  by  two  PSF  updating  iterations  (i.e.  n  =  2  in  Step  2). 

Fig.  18d  shows  the  initial  OTF  profile  for  Implementation  1  (note  the  cutoff  at  28).  The 
restored  object  at  the  end  of  the  10th  cycle  is  shown  in  Fig.  18e  and  the  reshaped  OTF  is  shown 
in  Fig.  3f  for  this  implementation.  It  is  evident  that  the  peaks  are  better  resolved,  and  more 
significantly,  the  frequency  range  is  extended  attesting  to  the  super-resolution  capabilities  of 
the  algorithm.  The  corresponding  performance  results  for  Implementation  2  are  shown  in  Figs. 
18g,  18h  and  18i,  which  display  the  initial  selection  of  OTF  profile  (with  cutoff  at  25),  the 
restored  object,  and  the  reshaped  OTF,  respectively.  The  robustness  of  the  algorithm  is  evident 
from  the  fact  that  the  performance  has  not  degraded  much  despite  the  larger  error  made  in  the 
selection  of  OTF.  A  greater  insight  into  the  robustness  feature  of  the  algorithm  can  also  be 
obtained  from  Figs.  18j  and  18k,  which  plot  the  variation  in  estimation  errors  (mean  square 
error  in  object  estimation  and  in  OTF  estimation)  as  the  number  of  cycles  of  algorithm 
implementation  is  increased. 
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Fig.  18j  FiS-  18k 


Fig.  18.  Demonstration  of  robustness  of  Blind  ML  Algorithm 


We  shall  now  describe  the  results  of  two  experiments  in  order  to  demonstrate  the 
performance  of  the  blind  ML  restoration  algorithm  in  estimating  both  the  object  an  the  sensor 
PSF  simultaneously.  In  the  first  experiment,  we  will  consider  a  simulated  one-dimensional 
object  while  in  the  second  experiment  we  will  consider  the  super-resolution  of  a  real  PMMW 
image. 

Starting  with  a  one-dimensional  object  consisting  of  three  pulses  of  uneven  heights, 
shown  in  Fig.  19a,  a  blurred  image,  shown  in  Fig.  19b,  was  obtained  by  convolving  with  a 
sensor  with  cutoff  frequency  14  and  a  OTF  whose  profile  is  shown  in  Fig.  19c.  For 
commencing  the  blind  ML  iterations,  an  initial  estimate  of  PSF  was  made  by  assuming  an  OTF 
with  cutoff  frequency  10  (an  error  was  made  deliberately  to  test  the  performance  of  the 
algorithm  to  yield  estimates  progressively  moving  towards  the  true  cutoff  frequency  of  17)  and 
taking  its  inverse  Fourier  transform.  The  assumed  OTF  and  the  PSF  are  shown  in  Figs.  19d 
and  19e. 

The  OTF  estimates  resulting  after  1  and  3  cycles  of  the  algorithm  are  shown  in  Figs. 
19f  and  19g  which  indicates  the  progression  of  the  algorithm  in  expanding  the  OTF  towards 
the  true  cutoff  frequency.  Each  cycle  of  algorithm  implementation  consists  of  five  object 
estimation  iterations  (m  =  5)  followed  by  two  PSF  updating  iterations  (n  =  2).  The  final 
results  after  10  cycles  of  algorithm  implementation  are  shown  in  Figs.  19h,  19i  and  19j,  which 
show  the  estimated  OTF,  the  corresponding  PSF  and  the  restored  object  respectively. 

In  the  second  experiment  we  tested  the  performance  of  the  blind  ML  restoration 
algorithm  in  processing  PMMW  images  supplied  by  the  Wright  Laboratory  Armament 
Directorate.  The  specific  image  used  as  input  to  the  algorithm  is  the  “Parking  Lot  3”  image 
which  is  shown  in  Fig.  20a.  For  obtaining  an  initial  estimate  of  the  PSF  to  commence  the 
iterations,  an  analysis  similar  to  the  one  described  earlier  of  matching  the  edge  profiles  was 
made  and  an  OTF  with  a  cutoff  frequency  of  82,  shown  in  Fig.  20b  (only  one  dimension  of 
OTF  is  shown  here,  for  simplicity),  was  developed.  The  ML  restoration  was  implemented  over 
20  cycles  with  5  ML  estimation  iterations  (m  =  5)  and  2  PSF  updating  iterations  (n  =  2)  in 
each  cycle.  The  restored  images  at  the  end  of  5  cycles  and  20  cycles  are  shown  in  Figs.  20c 


67 


and  20d.  The  reshaped  OTF  at  the  end  of  20  cycles  is  also  shown  in  Fig.  20e.  The  progressive 
enhancement  of  resolution  is  clearly  evident  from  the  improved  structural  details  of  the 
building  and  the  parked  automobiles. 

Fig.  20a  Fig.  20b 


Fig.  20c  Fig.  20d 


Figure  20.  Blind  ML  restoration  of  PMMW  image  (“ Parking  Lot  3  ”). 


4.2  Suppression  of  Noise  Induced  Artifacts 

It  is  well  known  that  image  restoration  algorithms  generally  produce  undesirable 
artifacts  because  of  the  ill-posed  nature  of  the  deconvolution  problem.  As  noted  in  the 
Introduction,  different  types  of  artifacts  are  generated  during  the  restoration  process  and  these 
are  due  to  various  causes.  In  the  development  of  super-resolution  algorithms  for  applications  in 
missile  seekers,  we  are  particularly  interested  in  noise  induced  artifacts  due  to  reasons  that  will 
be  elaborated  in  the  following  discussion. 

First  of  all,  it  should  be  emphasized  that  artifacts  generated  due  to  the  presence  of  noise 
in  the  image  data  manifest  themselves  in  the  restored  image  much  differently  from  the  more 
familiar  “ringing  artifacts”  [18].  The  latter  are  primarily  due  to  Gibbs  phenomena  resulting 
from  the  truncation  of  the  Fourier  components  caused  by  the  extent  of  the  acquired  spatial 
frequencies  of  the  scene  or  object  imaged  being  finite.  Consequently,  during  the  restoration 
process,  particularly  when  ML  estimation  methods  are  employed,  noticeable  distortions  near 
edges  occur  as  sharp  transitions,  or  edges  that  may  be  present  in  the  intensity  distribution 
function  become  more  accentuated  and  emerge  with  overshoots  (or  ringing).  For  our  present 
applications,  where  the  goal  of  restoration  is  to  assist  in  feature  extraction  operations,  ringing 
artifacts  which  essentially  show  up  as  periodic  (or  decaying  periodic)  repetitions  of  sharp 
intensity  transitions  in  the  image  may  not  be  objectionable.  However,  noise  induced  artifacts, 
which  could  show  up  with  more  general  patterns,  can  be  quite  troublesome  since  they  may 
obscure  some  critical  feature  of  the  target,  introduce  features  that  may  not  be  present  in  the 
scene,  or  otherwise  cause  distortions  of  the  feature  maps  that  will  be  extracted  from  the 

restored  image. 

In  super-resolution  processing,  where  the  primary  goal  is  the  creation  of  new 
frequencies  which  are  present  in  the  object  imaged  but  not  in  the  image  recorded,  the  artifacts 
resulting  from  the  presence  of  high  frequency  noise  need  to  be  given  special  consideration. 
Due  to  the  spectral  mixing  that  takes  place  during  the  execution  of  each  iteration,  it  is  likely 
that  the  high  frequency  components  of  the  noise  become  responsible  for  the  expansion  of  the 
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image  bandwidth,  which  show  up  as  artifacts  in  the  processed  image.  Suppression  of  such 
noise  induced  artifacts  is  hence  a  desirable  step  in  the  overall  processing. 


For  gaining  insight  into  how  the  presence  of  noise  affects  the  performance  of  the  blind 
ML  restoration  algorithm  discussed  in  the  previous  section,  a  controlled  experiment  was 
conducted  with  a  one-dimensional  object  blurred  with  a  known  PSF  function  and  restoration 
conducted  in  a  noise-free  scenario  as  well  as  when  a  significant  amount  of  noise  is  added.  Fig. 
21a  shows  the  object  with  three  spikes  that  was  blurred  with  a  OTF  whose  profile  is  shown  in 
Fig.  21b.  The  resulting  blurred  image  is  shown  in  Fig.  21c.  Processing  of  this  image  by  the 
blind  ML  restoration  algorithm  with  a  starting  selection  of  OTF  shown  in  Fig.  2 Id  results  in 
the  restored  image  at  the  end  of  10  cycles  of  algorithm  implementation  (with  m  =  5  and  n  = 
2)  shown  in  Fig.  21e,  and  the  reshaped  OTF  at  the  end  of  10  cycles  is  shown  in  Fig.  21f.  The 
experiment  was  repeated  with  the  addition  of  Gaussian  noise  creating  a  blurred  image  with 
SNR  =  11.2494  dB  shown  in  Fig.  21g.  Processing  of  this  image  with  the  same  starting 
selection  of  OTF  as  before  ( i.e .  OTF  shown  in  Fig.  21d)  resulted  in  the  restored  image  shown 
in  Fig.  21h  at  the  end  of  10  cycles,  and  a  reshaped  OTF  shown  in  Fig.  21i. 

The  significant  errors  introduced  in  OTF  estimation  in  the  processing  of  the  noisy 
image  are  particularly  noteworthy  and  illustrate  the  effects  of  the  added  noise.  Evidently,  some 
form  of  filtering  should  be  used  to  mitigate  these  effects  and  obtain  satisfactory  restoration 
performance.  Two  choices  are  available:  (i)  pre-filtering  of  the  noisy  image  prior  to  restoration 
processing,  and  (ii)  post  filtering  of  the  restored  image  to  correct  for  the  degradations  caused 
by  noise.  The  latter  appears  to  be  a  more  feasible  option  keeping  in  mind  the  goals  of 
achieving  improved  resolution  in  missile  seeker  applications,  since  pre-filtering  the  image 
while  successful  in  reducing  the  noise  contamination  also  reduces  the  resolution  in  the  image. 
In  restoration  studies,  it  is  generally  known  that  pre-filtering  methods  aimed  at  artifact 
reduction  extract  a  price  in  yielding  reduced  resolution  levels  in  the  processed  image. 
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Fig.  21a 


Fig.  21b 


Fig.  21c 


Fig.  21  g  Fig.  21  h  Fig.  21  i 


Fig.  21.  Results  of  processing  one-dimensional  signal  with  three  spikes 

(controlled  experiment) 

In  the  following,  we  shall  briefly  outline  two  post-filtering  adjustments  that  result  in 
modified  restoration  algorithms.  We  shall  refer  to  these  as  Blind  ML-PF1  and  Blind  ML-PF2 
algorithms  in  further  discussion.  The  commonality  between  these  is  to  adjust  the  OTF  function 
subsequent  to  the  PSF  estimation  step  (i.e.  Step  2  of  the  blind  ML  restoration  cycle)  by 
utilizing  the  low-pass  filtering  characteristic  of  physical  sensors  which  imposes  a 
monotonically  nonincreasing  behavior  on  the  OTF  magnitude  near  the  cutoff  frequency  as  well 
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as  typical  restrictions  on  the  width  of  passband  relative  to  the  sampling  frequency.  However, 
since  the  cutoff  frequency  is  not  known  a  priori,  and  the  goal  of  blind  ML  restoration  is  to 
progressively  extend  the  width  of  passband,  we  use  an  adjustment  procedure  suggested  from 
empirical  considerations  of  setting  the  OTF  magnitude  to  zero  for  all  frequencies  beyond  60  % 
of  the  folding  frequency  after  the  PSF  estimation  in  Step  2.  This  adjustment  results  in  the 
modified  algorithm  Blind  ML-PF1  which  updates  the  iterative  algorithm  given  in  the  previous 
section  by  modifying  Step  2  as  follows: 


Step  2*:  Using  object  estimate  fm(j)  obtained  at  the  end  of  Step  1,  implement  n 
iterations  of  PSF  updating  together  with  an  OTF  adjustment  step  by 

*0) 


,j  =  0,1,2,.  .  .  ,N 


(48) 


and 

Hm(co)  =  Hm(co),  o)  <  0.6fs/2  l  =  0,1,2,.  .  .  ,(n-l) 

=  0,  O)  >  0.6fs/2  (49) 

In  Equation  (49),  HM(a>)  denotes  the  DFT  of  hM(j)  J  =  1,  2,  .  .  .  ,N,  and  denotes  the 
sampling  frequency.  The  effect  of  the  adjustment  given  by  this  equation  in  compensating  for 
the  overextension  of  bandwidth  due  to  the  higher  frequency  components  of  noise  is  quite 

evident. 


An  alternative  approach  for  the  adjustment  of  OTF  results  in  improved  performance  in 
certain  cases  and  involves  replacing  the  adjustment  rule  in  Equation  (49)  by  the  following: 

HM(eo)  =  Hm(g>)  if  Hm(co)  <  Hm(o) - 1) 

=  0  if  HM(a>)>HM(a>)  (50) 

The  rationale  behind  the  adjustment  suggested  by  Equation  (50)  is  to  remove  the  possible 
ripples  in  the  OTF  outside  the  passband  contributed  by  the  higher  frequency  components  of 
noise  by  setting  the  OTF  magnitude  to  zero  if  it  starts  increasing  relative  to  its  value  at  the 
previous  frequency  step.  We  shall  hereafter  refer  to  this  algorithm  as  Blind  ML-PF2. 
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In  the  following  we  shall  briefly  present  results  of  some  experiments  to  highlight  the 
performance  improvements  obtained  from  the  modified  algorithms. 


Experiment  1:  In  this  experiment,  we  consider  the  restoration  of  the  one-dimensional 
object  with  three  spikes  that  was  discussed  earlier.  Processing  of  the  blurred  image  in  the 
noise-free  case,  which  is  shown  in  Fig.  21c,  with  the  same  starting  selection  of  OTF  as  before 
(shown  in  Fig.  2 Id)  results  in  the  restored  image  at  the  end  of  10  cycles  (with  m  -  5  and  n  = 
2)  of  implementing  Blind  ML-PF1  algorithm  which  is  shown  in  Fig.  22a.  The  reshaped  OTF 
in  this  case  is  shown  in  Fig.  22b.  The  corresponding  results  for  the  implementation  of  Blind 
ML-PF2  algorithm  are  shown  in  Figs.  22c  and  22d.  For  a  more  direct  comparison  with  the 
performance  of  the  original  algorithm,  we  show  in  Figs.  22e  and  22f  the  variations  in  the  mean 
square  error  in  object  estimation  and  in  OTF  estimation  as  the  iteration  cycles  progress  for  the 
original  algorithm  and  for  Blind  ML-PF1  and  Blind  ML-PF2  algorithms.  These  sketches 
indicate  very  little  changes  in  the  estimation  errors  for  the  various  algorithms  in  this  case  of 
processing  the  noise-free  signal.  However,  considering  now  the  restoration  processing  of  the 
noisy  blurred  image  shown  in  Fig.  21g  (with  SNR  =  11.2494  dB),  with  the  same  starting 
selection  of  OTF  as  before,  Blind  ML-PF1  algorithm  results  in  a  restored  image  shown  in  Fig. 
22g  and  a  reshaped  OTF  shown  in  Fig.  22h  at  the  end  of  10  cycles,  whereas  Blind  ML-PF2 
algorithm  results  in  a  restored  image  shown  in  Fig.  22i  and  a  reshaped  OTF  shown  in  Fig.  22j. 
More  dramatic  is  the  comparison  of  mean  square  errors  in  this  case.  Figs.  22k  and  221 
compare  the  MSE  in  object  estimation  and  in  OTF  estimation  for  the  three  algorithms.  The 
improvements  resulting  from  the  modified  algorithms  clearly  illustrate  the  compensation  for 
the  effects  of  noise  provided  by  the  two  modifications  suggested  here. 
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Fig.  22i 


Fig.  22j 


Fig.  22k 


Fig.  221 


Fig.  22  (continued).  Results  of  processing  with  Blind  ML-PF1  and  Blind  ML-PF2  Algorithms 


Experiment  2:  The  performance  of  the  modified  algorithms  in  processing  noisy  images 
was  also  tested  by  processing  a  256x256  image  from  our  database.  Fig.  23a  shows  the  original 
image  used  in  this  experiment.  Fig.  23b  shows  the  blurred  image  obtained  by  convolution  with 
the  PSF  of  a  sensor  with  cutoff  coc  =  63  and  addition  of  Gaussian  noise.  The  SNR  in  this 
image  is  10.1084  dB.  The  reconstructed  image  after  6  cycles  of  processing  with  the  original 
blind  ML  algorithm  (with  m  =  5  and  n  =  2)  is  shown  in  Fig.  23c,  which  shows  a  marked 
improvement  in  the  resolution.  However,  some  artifacts  due  to  the  significant  amount  of  noise 
present  in  the  input  image  are  clearly  visible  in  the  restoration  (see  the  dark  line  under  the  left 
eye,  for  illustration).  Fig.  23d  shows  the  restored  image  after  6  cycles  of  processing  with  Blind 
ML-PF1  algorithm,  which  shows  that  the  resolution  enhancements  are  retained  while  some  of 
the  noise-induced  artifacts  are  successfully  eliminated.  The  frequency  extrapolation  resulting 
from  this  algorithm  to  achieve  superresolution  can  also  be  demonstrated  by  comparing  the 
frequency  spectra  of  the  input  blurred  image  (shown  in  Fig.  23f)  and  of  the  restored  image 
(shown  in  Fig.  23g)  with  the  spectrum  of  the  original  image  (shown  in  Fig.  23e).  The 
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4.3  Modified  ML  Algorithm  With  Background-Detail  Separation 

Since  the  primary  goal  of  super-resolution  processing  is  to  enhance  the  resolution  in  the 
processed  image  through  the  creation  of  spatial  frequencies  of  the  object  that  are  absent  in  the 
acquired  image,  an  important  measure  of  the  performance  of  a  super-resolution  algorithm  is 
the  extent  of  spectral  extrapolation  it  can  provide.  It  has  become  quite  well-known  [11,13,28] 
that  the  amount  of  bandwidth  extension  beyond  the  cutoff  frequency  an  algorithm  can  provide 
depends  on  certain  characteristics  of  the  image  being  processed,  most  notably  the  spatial  extent 
of  the  object  in  the  scene  imaged.  Experimental  researchers  have  noted  frustrating  experiences 
of  not  being  able  to  duplicate  for  their  own  images  the  bandwidth  extension  claims  made  by  the 
developers  of  specific  algorithms  with  carefully  crafted  examples  of  specifically  tailored 
images.  In  particular,  it  has  now  become  common  knowledge  that  while  very  impressive 
bandwidth  extensions  are  possible  for  images  with  spatially  limited  point  sources  against  a 
uniform  zero  background  (such  as  those  arising  in  astronomical  applications),  spectral 
extrapolation  to  similar  degrees  cannot  be  obtained  in  general  for  images  that  contain  objects 
which  are  not  spatially  limited,  Unfortunately,  a  majority  of  images  one  encounters  in  military 
applications,  particularly  in  target  surveillance  and  tracking  scenarios,  fall  into  this  latter 
category  and  typically  contain  several  spatially  distributed  objects  of  interest  (tanks,  aircraft, 
military  vehicles,  etc.)  sharing  the  scene  with  some  background  which  may  or  nay  not  be 
uniform. 

It  has  also  been  observed  by  some  image  processing  researchers  [31,32]  that  separation  of 
the  background  from  the  detail  component  of  the  object  imaged  and  subjecting  these  two 
components  to  different  forms  of  processing  can  yield  significant  benefits.  In  particular,  if  the 
object  being  imaged  permits  a  separation  of  these  two  components  such  that  the  background  is 
uniform  and  extends  over  the  whole  extent  of  the  scene,  while  the  detail  component  is  sparsely 
distributed  (or  nearly  sparsely  distributed),  one  can  subtract  the  smooth  background  and 
attempt  to  restored  only  the  detail  component.  Of  particular  significance  to  super-resolution 
processing  from  this  strategy  is  the  possibility  of  exploiting  the  spatially  limited  structures  that 
may  become  available  when  one  focuses  on  only  the  detail  component.  It  is  hence  possible  to 
develop  optimized  algorithms  that  provide  an  enhanced  reconstruction  of  edges  and  other 
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features  of  targets  of  interest  while  suppressing  the  ringing  artifacts  these  structures  typically 
generate  in  the  restoration  process. 

While  the  idea  of  background-detail  separation  outlined  above  seems  conceptually  simple,  a 
number  of  intricate  issues  need  to  be  addressed  for  a  satisfactory  implementation  of  a  ML 
super-resolution  algorithm  utilizing  this  idea.  First  of  all,  due  to  the  subtraction  of  the 
background,  the  detail  component  will  not  be  nonnegative  at  all  pixel  locations  and 
consequently  appropriate  measures  (thresholding,  for  instance)  need  to  be  introduced  to 
enforce  non-negativity  constraints.  Secondly,  since  the  detail  component  contains  all  the  high 
frequency  portions  of  the  image  (point  sources,  sharp  edges,  high  frequency  noise,  etc.), 
growth  of  artifacts  resulting  from  ML  iterations  of  these  need  to  be  curtailed.  A  typical 
processing  step  that  accomplishes  this  objective  of  suppressing  the  ringing  (or  the  halo  effect 
near  sharp  edges)  is  to  introduce  a  soft  thresholding  operation  [31,33]  of  the  form 


where  a  and  p  are  appropriately  chosen  constants,  after  each  iteration  of  obtaining  the  ML 

estimate  p  .  Finally,  since  the  background  component,  which  is  processed  independently,  will 
be  added  to  the  estimate  of  the  detail  component,  which  is  subjected  to  a  nonlinear  operation  of 
the  above  form,  in  order  to  form  the  complete  image  estimate,  appropriate  constraints  to 
ensure  conservation  of  the  total  energy  of  the  image  [32]  are  to  be  introduced. 

While  several  different  algorithms  can  be  constructed  [33]  by  employing  alternate 
procedures  for  enforcing  the  needed  constraints  as  discussed  above,  we  shall  briefly  outline  a 
specific  algorithm  that  has  yielded  superior  performance  especially  in  the  context  of  processing 
PMMW  images.  The  development  of  the  algorithm  proceeds  with  rewriting  the  model  for  the 
imaging  process  given  by  Equation  (1)  as 

g{y)  =  X  h(y,  x)[fb  {x)+fd  (*)]+  noise  (52) 

xeX 
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where  fd{x)  denotes  the  detail  component  and  fb(x)  denotes  a  smooth  background 
component. 


Since  the  background  is  smooth,  the  estimate  of  this  component  is  relatively  invariant  to  the 
effects  of  convolution  with  the  sensor  PSF  and  hence  the  estimate  fb(j )  for  a  discretized 
image  can  be  obtained  by  severely  blurring  the  image  data,  i.e. 

f  (j)=h{j)®g(j),  j  =  1,2, ...,N  (53) 

where  N  is  the  total  number  of  pixels  in  the  image.  A  ML  estimate  of  the  detail  component  is 

now  obtained  iteratively  using  the  initial  value  f Q{j)  =  g{j)~f  if)  and  progressively 
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constructing  the  estimates  fk  {j)  by 
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j  =  1,2,...  ,N 

For  suppressing  the  artifacts,  a  soft  threshold  constraint  is  applied  after  the  execution  of  each 
iteration  by  computing 
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with  selected  values  of  the  constants  a  and  ft  .  The  estimate  of  the  complete  image  at  this 
iteration  is  now  computed  as 


( „  d 


fn+\(f)=c  /»+i0‘)  \  +  fb(f) 


(56) 


and  constraints  to  enforce  non-negativity  and  energy  conservation  are  applied.  The  non¬ 
negativity  constraint  can  be  implemented  simply  from  a  thresholding  operation, 
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(57) 


/„,(/)< °.  =>  set  /»+i0')=° 

/n+1  O')  *  0  .  =  >  Set  /-I  O')  =  fn+x  0)  ’ 

while  the  energy  conservation  constraint  can  be  implemented  by  a  scaling  operation 

7=1 

The  initial  value  for  performing  the  next  ML  iteration  is  now  obtained  as 

A^/\  A  /\  A^/\ 

O') 

and  the  algorithm  is  repeated  by  executing  the  operations  described  by  Equations  (54)-(59). 

While  the  ML  estimation  performed  in  Equation  (54)  is  similar  to  that  in  the  basic 
algorithm  (given  by  Equation  (24)),  the  additional  steps  described  by  Equations  (55)-(59)  result 
in  an  optimized  version  that  produces  further  improvements  in  resolution  together  with 
suppression  of  ringing  artifacts.  We  will  refer  to  this  algorithm  as  ML-BD  algorithm  in  later 
discussion.  These  operations,  however,  are  not  computation  intensive  and  hence  the  processor 
requirements  for  implementing  the  algorithm  do  not  change  significantly.  For  a  progressive 
correction  of  the  sensor  PSF  h(j)  along  with  the  estimation  of  the  object,  a  blind  version  of  this 
algorithm  can  be  implemented  by  including  an  updating  of  the  PSF  at  the  end  of  each  object 
estimation  iteration  as  in  Equation  (47),  resulting  in  a  Blind  ML-BD  algorithm. 

A  few  words  on  the  selection  of  parameters  a  and  p  for  implementing  the  soft  threshold 
constraint  given  by  Equation  (55)  seems  to  be  in  order,  These  parameters  need  to  be  selected 
with  care  to  obtain  optimized  results  that  ensure  achieving  the  twin  objectives  of  resolution 
enhancement  and  artifact  suppression  (which,  as  noted  earlier,  are  often  contradictory).  Values 
of  these  parameters  can  be  tailored  to  the  specific  class  of  images  being  processed  from 
conducting  simple  preliminary  analysis  steps.  For  instance,  in  the  selection  of  appropriate 
values  for  these  parameters  for  processing  images  containing  man-made  objects  (such  as  the 
PMMW  image  “Humvee”),  one  may  note  that  combinational  geometry  models  are  often 
appropriate  for  defining  such  objects  and  a  useful  description  of  each  object  can  be  obtained 
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from  a  combination  of  eight  solid  geometric  primitives:  rectangular  parallelepiped,  box, 
sphere,  right  circular  cylinder,  right  elliptical  cylinder,  truncated  right  angle  cone,  ellipsoid  of 
revolution,  and  right  angle  wedge  [34],  By  selecting  the  appropriate  primitives  and  combining 
them,  the  shape  and  the  location  of  any  object  can  be  described.  For  instance,  to  model  the 
military  vehicle  in  the  PMMW  image  “Humvee”,  one  may  observe  that  the  four  primitives, 
rectangular  parallelpiped,  box,  right  angle  wedge  and  right  circular  cylinder  are  sufficient. 
When  mapping  this  3-D  model  on  a  2-D  plane,  the  image  can  be  viewed  as  a  superposition  of 
several  one-dimensional  blurred  pulse  signals.  Thus  an  analysis  for  recovering  the  edges  from 
these  blurred  pulses  can  be  conducted  and  optimal  values  of  a  and  /?  can  be  determined  for 
implementing  the  ML-BD  algorithm  to  ensure  satisfactory  edge  recovery  with  minimal  ringing 
artifacts.  Following  this  analysis,  the  values  of  a  and  /?  were  determined  as  a  =  1  and  ft  - 
-0.0005  which  can  further  be  used  for  processing  the  two-dimensional  image. 

Figs.  24a  and  24b  show  the  result  of  processing  the  PMMW  image  “Humvee”  (acquired 
image  shown  in  Fig.  4a)  with  ML-BD  algorithm  at  the  end  of  3  iteration  cycles.  Comparing 
with  the  results  earlier  presented  (Fig.  10),  it  may  be  noted  that  an  improved  resolution  is 
achieved  (which  is  evident  from  the  details  on  the  body  of  the  vehicle  and  the  sharper  edges) 
while  restoration  artifacts  are  considerably  eliminated  (as  evident  from  the  restoration  of  the 
windshield  and  of  the  undercarriage  of  the  vehicle). 


Fig.  24a  Fig.  24b 


Fig.  24.  Results  of  processing  “Humvee”  image  by  ML-BD  algorithm 
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4.4  Modified  Algorithm  for  Maximization  of  Posterior  Density  Function 

The  Maximum  Likelihood  (ML)  restoration  algorithm  described  by  Equation  (24) 
attempts  to  solve  the  inverse  problem  underlying  the  image  restoration  and  super-resolution 
goals  by  employing  a  statistical  modeling  of  the  imaging  process  in  order  to  optimize  the 
“likelihood  function”.  In  certain  cases  when  a  mathematical  model  can  be  formulated  for  the 
prior  distribution  characterizing  the  object  being  estimated,  which  in  the  case  of  image 
restoration  is  the  probability  distribution  of  the  intensity  emanating  from  the  object  or  the  scene 
being  imaged,  it  is  possible  to  utilize  this  additional  information  to  devise  algorithms  that 
provide  improved  performance  as  originally  shown  by  Hunt  [35].  Fundamental  to  this 
approach  is  a  consideration  of  the  posterior  distribution  as  the  quantity  to  be  optimized  (instead 
of  the  likelihood  function  which  ML  algorithms  attempt  to  optimize).  For  an  image  formation 
process  modeled  by  an  input-output  model  of  the  form 

g(x,y)  =  3(/(x,>0)  <60> 

where  f(x,y )  denotes  the  intensity  function  of  the  object  being  imaged  defined  over  a  region 
(x,y)sR0  and  g{x,y)  denotes  the  intensity  detected  in  the  image  over  a  region  {x,y)eR, ,  Bayes 
rule  relates  these  distributions  in  the  form 


p{f/g)  = 


^(g//)p(/) 

p{s) 


(61) 


In  Equation  (61),  p(f/g )  denotes  the  posterior  density  function  and  p(g/ f)  denotes  the 

likelihood  function,  which  are  related  through  the  prior  distribution  p(f ) .  It  must  be  noted 
that  with  a  statistical  description  of  the  imaging  operation  given  by  Equation  (60),  /  denotes  a 
set  of  random  variables  whose  distribution  p(f)  portrays  the  distribution  followed  by  the 
photons  right  after  they  bounce  off  the  object  being  imaged  and  before  they  travel  to  the 
imaging  sensor.  Thus,  p(f)  summarizes  the  prior  knowledge  about  the  object  and  hence  is 
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called  the  “prior  density”  (or  simply  the  “prior”).  Consequently,  making  use  of  this  knowledge 
can  steer  the  optimization  process  in  the  right  direction  and  can  in  turn  result  in  improved 
object  estimates. 

In  MAP  estimation,  one  attempts  to  find  the  estimate  f  that  maximizes  p(f  / g) ,  i.e. 


f  =  arg  max  p(f  /  g) .  (62) 

Since  p(g)  in  Equation  (61)  is  independent  of  f ,  this  problem  reduces  to  finding 


/  =  arg  ma x[p(g// )p(f )]  •  (63) 

Thus,  depending  on  the  statistical  model  used  for  p{g/f )  and  p{f) ,  different  estimates  f 
can  be  obtained  by  solving  this  maximization  problem. 


While  different  distribution  models  could  be  used  for  modeling  the  prior  p{f) ,  a 
Poisson  distribution  function  results  in  a  particularly  simple  algorithm.  In  this  case  p(f)  is 
modeled  as 


p(/W))  =  n 

*,y 


f(x,y) 


(64) 


where  f(x,y)  denotes  the  mean  photon  emission  rate  at  the  location  (x,y).  As  in  the 
derivation  of  the  ML  restoration  algorithm  given  earlier,  the  likelihood  function  is  also 
modeled  by  a  Poisson  distribution 
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p(g/f)= n 
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(65) 


One  may  now  proceed  to  solve  the  maximization  problem  posed  in  Equation  (62)  by 
taking  the  derivative  of  p{f/g )  and  setting  it  equal  to  zero.  As  in  the  derivation  of  the  ML 
algorithm,  considerable  simplification  can  be  obtained  by  attempting  to  maximize  the  natural 
logarithm  of  p(f/g)  ■  Thus,  a  MAP  estimate  f  is  sought  by  noting  that 


In  p{flg)  =  In  p(g/f)  +  ln^(/)  -  In  p(g) 
which  results  in  the  extremum  condition 


(66) 


d  In  p(f /g)  d  \np{g!f)  d  In  pij) 

Jf  =  df  +  df 


(67) 


Employing  the  assumed  distributions  for  p{g/f)  and  p{f)  and  using  a  simplification  afforded 
by  Stirling’s  approximation 


lnz!wzlnz-z  (^^) 

one  may  evaluate  the  derivative  on  the  Left  Hand  Side  of  Equation  (67)  as 
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d  ^p{f  Is) 

df 


^(/(x\y)in/(x^y)-/(v,y)-/(ac\y)in/(^,,y)+/(x,,y)) 
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d  f{x,y)  L' 

3  ^ ( g(x  ,y)  In (h  ®  /)(*' , y  )  -  (h  ®  f)(x' , y  )  -  g(x' , y  )  In g(x' , y  )  +  g(x  , y  )) 


d  f{x,y) 


\x,y 


-  '"^y)  -tof(x-y) + ((a®/)oL)  “  T  "M 


(69) 


where  the  symbol  *  denotes  correlation. 

Setting  this  quantity  equal  to  zero,  one  now  obtains  the  extremum  condition  in  the 
simplified  form 


Equation  (70)  is 
form 


lnf(x,y)-\nf(x,y)- 


g{x,y) 


( h®f)(x,y ) 


<h(x,y)  =  0 


(70) 


still  complicated  and  hence  one  attempts  to  obtain  an  iterative  solution  in  the 


fn+'(x,y)  =  F(fn(x,y))  (71) 

where  F(.)  is  the  update  function. 

Taking  the  exponential  on  both  sides  of  Equation  (70)  and  rearranging  the  terms,  one 
now  obtains 


f{x>y)  ~  f{x>y)exP 


(f 


g(*>y) 


vv 


(/i®/)(x,y) 


-1 


‘h(x,y) 


(72) 


By  making  the  assertion  that  f{x,y)  is  equal  to  the  current  estimate,  the  iterative  updating  rule 
for  constructing  MAP  estimates  is  finally  obtained  in  the  form 
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/n+1(x,^)  =  /n(x^)exp 


(73) 


(f 


g(x,y) 


(h<S>fn)(x,y) 


fh(x,y) 


In  order  to  start  the  iterative  scheme,  f°(x,y)  =  g{x,y)  is  normally  assumed. 


It  is  interesting  to  note  that  the  iterative  algorithm  specified  by  Equation  (73)  bears  a 
close  resemblance  to  the  ML  algorithm  described  by  Equation  (24).  The  principal  difference  is 
the  presence  of  the  exponential  term  in  the  MAP  algorithm  which  provides  an  additional 
nonlinear  processing  term  to  permit  generation  of  new  frequencies  thus  facilitating  improved 
spectral  extrapolation  in  some  cases.  On  the  negative  side,  the  presence  of  this  term  precludes 
making  general  assertions  regarding  the  convergence  of  the  iterations.  Nevertheless, 
convergence  has  been  noted  in  simulations,  with  even  superior  restoration  results  compared  to 
the  ML  algorithm  in  several  numerical  experiments.  As  in  the  case  of  the  ML  algorithm,  the 
speed  of  convergence  is  governed  by  several  factors,  most  notably  the  accuracy  of  the 
knowledge  of  PSF  and  the  SNR  of  the  image.  The  MAP  algorithm  seems  to  have  less 
robustness  to  PSF  variations  than  the  ML  algorithm. 

For  a  further  comparison  of  the  MAP  algorithm  with  the  more  familiar  ML  algorithm, 
it  is  interesting  to  note  that  under  certain  conditions,  the  iterations  specified  by  the  updating 
rule  given  by  Equation  (73)  approximate  the  updating  executed  in  the  implementation  of  the 
ML  algorithm.  Specifically,  we  observe  that  exp(/?)  can  be  approximated  by  the  first  two 
terms  in  its  power  series  expansion,  viz.,  1  +  /?,  when  the  magnitude  of  /3  is  small.  Hence, 
when  the  numerical  values  of 


P(x,y)  = 


g(x,y) 


h®fn{x,y ) 

attain  small  magnitudes,  the  Right  Hand  Side  of  Equation  (73)  can  be  replaced  by 


(74) 


f"(x,y) 


g(x,y ) 


\_h®fn(x,y)\ 


■h(x,y) 


(75) 
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without  introduction  of  much  error.  In  this  case,  the  updating  required  by  the  MAP  algorithm 
will  be  identical  to  that  required  by  the  ML  algorithm.  One  may  also  note  that  0  (x,y)  in 
Equation  (74)  gives  a  measure  of  the  residual  error 

e(x,y)  =  g(x,y )  -h®fn  (x,y)  (76) 

at  the  end  of  the  n,h  iteration.  Thus  the  performance  of  the  MAP  algorithm  will  approximately 
track  the  performance  of  the  ML  algorithm  after  the  completion  of  a  sufficient  number  of 
iterations  when  the  residual  error  will  hopefully  have  reduced  to  rather  small  values.  However, 
this  need  not  be  the  case  in  the  initial  steps  of  the  algorithm  implementation  when  the  residual 
errors  could  have  significantly  large  magnitudes. 

It  needs  to  be  emphasized  that  successful  restoration  and  super-resolution  using  MAP 
estimation  approach  can  greatly  depend  on  the  question  -  which  model  to  use  for  the  prior? 
Geman  and  Geman  [36]  suggest  the  use  of  Markov  Random  Field  (MRF)  for  reconstructing 
prior  models  which  provide  the  ability  to  describe  spatial  correlation  in  the  object  intensity 
function.  In  a  Markov  model,  each  pixel  is  envisioned  to  loosely  belong  to  some  connected  set 
of  pixels  (called  “cliques”)  and  a  MRF  prior  can  be  formulated  to  specify  any  known 
information  in  the  reconstruction  by  exploring  the  connectivity  of  pixels.  Gibbs  functions 
typically  provide  a  powerful  representation  for  MRF  priors.  The  possibility  of  developing 
simple  iterative  updating  rules  such  as  that  given  by  Equation  (73)  may  however  diminish  with 
the  use  of  such  complex  (albeit  more  accurate)  prior  functions  in  the  maximization  process, 
leaving  one  to  settle  for  numerical  optimization  procedures.  Furthermore,  one  may  not  be  able 
to  make  any  predictions  on  the  convergence  behavior  of  such  implementations. 

As  mentioned  earlier,  restorations  with  the  MAP  algorithm  in  general  yield  very  similar 
results  comparable  to  the  use  of  the  ML  algorithm.  In  specific  cases  one  may  expect  to  see 
some  slightly  improved  performance  however.  An  experiment  that  was  conducted  to  evaluate 
the  performance  of  the  Map  algorithm  will  be  outlined  now.  Fig.  25a  shows  a  PMMW  image 
acquired  by  the  ARL  95  GHz  camera  with  a  3  inch  lens/hom.  This  is  the  image  (“pattern 
target”)  obtained  at  a  range  of  about  200  inches  of  a  4  ft  x  8  ft  specially  constructed  target 
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board  with  rectangular  patches  of  radiation  absorbing  material.  The  restored  image  after  60 
iterations  of  processing  with  the  ML  algorithm  is  shown  in  Fig.  25b.  The  enhancement  of  the 
edges  and  the  sharpening  of  the  features  of  the  geometrical  patterns  are  clearly  visible  in  the 
processed  image.  The  corresponding  restoration  obtained  by  the  use  of  MAP  algorithm  is 
shown  in  Fig.  25c  which  depicts  the  result  after  60  iterations  of  processing. 


Fig.  25a.  PMMW image  “pattern  target” 


Fig.  25b.  Restoration  by  ML  algorithm 


Fig.  25c.  Restoration  by  MAP  algorithm 
Figs.  25a-c.  Super-resolution  of  the  “Test  Pattern”  image  by  MAP  Algorithm 
The  restorations  shown  in  Figs.  25b  and  25c  appear  almost  identical  at  first  glance. 
Some  quantitative  calculations  can  be  made  to  compare  the  results.  It  should  be  emphasized 
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that  unlike  in  the  case  of  the  computation  of  resolution  enhancement  outlined  earlier,  a 
comparison  with  the  ideal  object  is  not  possible  in  this  case  to  calculate  a  parameter  such  as  the 
virtual  aperture  diameter.  Consequently,  a  more  subjective  evaluation  may  be  needed  in  this 
case.  One  viable  approach  suggested  in  [41]  is  to  extract  a  slice  of  data  from  the  spatial 
frequency  distribution  plots  for  the  processed  and  the  unprocessed  images  and  compare  them. 
Fig. 26  shows  the  variation  of  the  normalized  intensity  (in  dB)  plotted  as  a  function  of  the 
spatial  frequency  for  a  cut  through  the  spectrum  of  the  original  acquired  image  in  Fig.  25a 
(shown  in  darker  line)  and  also  the  corresponding  variation  for  the  ML  processed  image  in  Fig. 
25b  (shown  in  fainter  line).  For  specified  intensity  thresholds,  one  may  identify  the 
corresponding  pixel  numbers  in  the  processed  and  the  original  data  and  compute  the 
improvement  by  evaluating  the  ratio  pixel(processed)  /  pixel  (unprocessed) .  These 
improvement  factors  computed  at  a  few  representative  intensity  levels  are  tabulated  below  for 
illustrative  purposes. 


Intensity  level 

(dB) 

Pixel  Unprocessed 

Pixel  Processed 

Improvement 

Factor 

-40 

8 

18 

2.25 

-50 

12 

32 

2.66 

-60 

20 

32 

1.60 
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20398B,  Unprocessed  and  Processed 


Fig.  26.  Comparison  of  spatial  frequency  spectra  of processed  (ML)  and  unprocessed  images. 

Corresponding  values  of  the  Improvement  Factor  can  be  computed  from  the  MAP 
processed  image  (in  Fig.  25c)  at  the  same  intensity  levels  for  a  comparison.  These  values  are 
shown  tabulated  below.  The  enhanced  values  for  the  MAP  algorithm  show  that  a  somewhat 
improved  restoration  has  been  achieved  in  this  case. 


Intensity  level 

(dB) 

Pixel  Unprocessed 

Pixel  Processed 

Improvement 

Factor 

-40 

8 

22 

2.75 

-50 

12 

35 

2.92 

-60 

20 

34 

1.70 
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5.  SELECTION  OF  SAMPLING  RATES  AND  DESIGN  OF  OPTIMIZED  SUPER¬ 
RESOLUTION  ALGORITHMS  WITH  UPSAMPLING  OPERATIONS 

An  aspect  of  particular  importance  in  the  digital  implementation  of  restoration  and 
super-resolution  algorithms  is  the  size  of  the  sampling  grid  used  for  the  representation  of 
image  data.  Sampling  at  the  Nyquist  rate  by  a  selection  of  the  sampling  pitch  px  =  py  =  nla>c, 

where  o)c  is  the  cut-off  frequency  of  the  diffraction-limited  imaging  sensor,  corresponds  to  an 
optimal  spacing  of  detector  elements  or  using  a  scan  rate  that  provides  the  largest  dwell  time 
(thus  enabling  an  increased  SNR  in  the  acquired  image).  However,  selection  of  this  grid  size 
can  lead  to  some  problems  when  the  Nyquist  sampled  image  is  subjected  to  super-resolution 
processing  since  the  extrapolated  frequencies  will  cause  an  overlap  with  the  passband 
frequency  components  leading  to  aliasing  of  the  spectrum.  Consequently,  one  needs  to  work 
with  oversampled  images  which  provide  a  region  of  support  for  the  creation  of  new 
frequencies  such  that  these  will  not  overlap  with  the  existing  frequency  components.  Sampling 
grid  selection  is  hence  a  very  important  design  consideration.  In  this  section  we  shall  outline 
our  investigations  on  this  issue  which  has  led  to  the  development  of  a  progressive  upsampling 
scheme  for  optimal  implementation  of  super-resolution  algorithms. 

5.1  Image  Representation  on  a  Sample  Grid 

Image  data  typically  used  in  processing  operations  will  be  in  a  sampled  form.  This 
could  be  a  resultant  of  the  hardware  limitations  in  the  data  acquisition  process  which  permit  the 
imaging  sensor  to  record  the  intensities  only  in  a  discrete  form  or  it  could  be  motivated  by  the 
desire  for  digital  processing  of  data  by  using  computer-based  algorithms.  The  timing  circuitry 
that  controls  the  sampling  of  energy  in  the  focal  plane  of  the  sensor  is  usually  a  very  important 
part  of  the  overall  data  acquisition  hardware.  For  scan-type  focal  plane  imaging  sensors,  the 
timing  circuitry  controls  the  acquisition  of  samples  and  also  automatically  adjusts  the  dwell 
time  to  match  the  scan  velocity  thus  controlling  the  frame  rate  of  the  acquisition  process. 
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The  signal  that  represents  the  object  or  the  scene  being  imaged,  denoted  f(x,y)  in 

Equation  (1),  is  a  continuous  signal  defined  on  a  region  (x,y)sR0  and  the  spectrum  of  this 

signal  typically  will  have  frequency  components  ranging  from  0  to  oo.  This  signal  when 
subjected  to  a  low-pass  filtering  operation  corresponding  to  diffraction  limited  imaging  with  a 

sensor  with  cutoff  frequency  coc ,  results  in  an  image  g(x,y)  which  is  also  a  continuous  signal 
defined  over  a  region  ( 'x,yjsRj .  Due  to  convolution  with  the  PSF  of  the  sensor,  the  spectrum 
of  g(x,y)  is  band  limited  to  the  range  0  <  \co\  <  a>c ,  however. 

As  noted  before,  the  recorded  image  data  is  in  the  sampled  form.  Let  gs(x,y)  denote 
the  sampled  version  of  the  image  and  let  px  and  py  denote  the  sampling  pitches  in  the  x-  and  y- 
directions  respectively.  Ideally  the  image  sampling  process  can  be  characterized  as  modulating 
the  signal  g{x,y)  with  a  set  of  impulse  functions.  Hence,  denoting  by  M  and  N  the  number  of 
samples  collected  in  the  jc-  and  y-directions  respectively,  the  sampled  signal  can  be  described 
by 


MN  v 

gt(*,y)  =  g  (*>y)LTts(x-mPx>y-”Py) 

m=  1  n= 1 


which  can  be  rewritten  as 


(77) 


M  N  .  v  /  \ 

gs{x,y)  =  ZZs  \mPx >nPy)S  (*  -  MPx >y  -  nPy)  (78) 

m- 1  n=\ 

where  the  set  of  values  g(mpx,npy},  m=l,2,3,...M  and  n  =  l,2,3,...N  constitute  the  image 
samples.  For  a  concise  representation,  the  sampling  pitches  px  and  py  can  be  dropped  and  the 
sampled  image  can  be  described  as 


g(m,n)  =  g  (mpx,npy) . 


(79) 


It  should  be  emphasized  that  g{m,n)  now  denotes  the  sampled  version  of  the  original 
continuous  signal  g(x,y) . 

To  examine  the  effect  of  the  sampling  operation  on  the  image  spectrum,  taking  the 
Fourier  transform  of  Equation  (78)  one  obtains 


G.(jat,jar) 


-kjQ:,ja,-!)Sry) 


(80) 


where  Q*  -2n  /  px  and  fi*  =  In  /py  denote  the  sampling  frequencies  (in  rad/m)  in  the  x-  and 
y-directions.  Equation  (80)  implies  that  the  spectrum  of  the  sampled  image  g(m,n)  is  made  up 
of  superimposed  and  shifted  copies  of  the  spectrum  of  the  nonsampled  original  image  g(x,y) . 
If  is  band-limited  and  the  bandwidth  extends  only  to  a  finite  frequency  value  coh ,  and  a 

sufficiently  high  rate  of  sampling  is  used  to  prevent  an  overlap  of  the  shifted  copies  in 
G  (jQx,jQy) ,  a  reconstruction  of  the  continuous  image  is  possible  from  a  low  pass  filtering  of 

the  sampled  image.  The  low  pass  filter  is  applied  to  the  sampled  image  such  that  only  one  copy 
of  the  spectrum  of  the  original  continuous  image  is  preserved  while  the  rest  (which  are 
repititions  of  this  portion)  are  discarded,  which  results  in  the  reconstructed  signal 


8r{x,y)=  E  'Eg(kj)hr(X-kP*’y-lPy)  (81) 

k~— oo  l — — <20 

In  Equation  (81),  gr(x,y)  denotes  the  reconstructed  continuous  signal  and  h\x,y)  denotes  the 

spatial  response  of  an  ideal  low  pass  filter  centered  at  (0,0)  in  the  frequency  domain  with  a 
zero  frequency  gain  equal  to  pxpy  and  cutoff  frequencies  -  n/px ,  njpx ,  -xjpy ,  and  njpy 
respectively.  A  reconstruction  filter  that  achieves  this  objective  is  given  by 


94 


(82) 


jtx  .  n  y 

sin - sm— — 

_ Px_ _ Ei_ 

nx  ny 

Px  Py 

For  a  band-limited  image  with  the  highest  spatial  frequency  coh,  Nyquist  theorem 
ensures  that  if  the  sampling  frequencies  are  selected  such  that  Q*  >  2 coh  and  £2*  >  2coh ,  there 

is  no  overlap  of  the  spectral  components  in  the  sampled  image.  Images  obtained  from 
diffraction-limited  sensing  are  necessarily  band-limited  to  the  sensor  cutoff  frequency  coc . 
Hence  no  aliasing  occurs  if  the  sampling  pitches  are  selected  to  satisfy  the  relations 

px<  n  I  co c  and  py<nlac.  (83) 

Evidently,  selection  of  the  largest  allowable  value  for  px  and  py ,  viz.,  px  —  py  =  it ! ooc ,  results 

in  the  optimal  number  of  detector  elements  to  be  used  in  the  data  acquisition  process  and  also 
results  in  an  image  of  a  correspondingly  small  size,  which,  is  attractive  from  computational 
considerations.  The  sampling  rate  in  this  case  is  referred  to  as  the  Nyquist  rate. 

For  illustrating  the  efficiency  of  representing  an  image  on  a  selected  grid  of  samples 
and  the  corresponding  frequency-domain  properties,  let  us  again  consider  the  “concentric 
disks”  image  described  earlier  in  Section  3.2.  For  convenience,  this  is  shown  again  in  Fig. 
27a.  This  is  a  simulated  image  of  a  series  of  concentric  disks  with  the  background  (dark)  at 
intensity  value  zero  and  the  disks  (bright)  at  intensity  value  one,  and  is  represented  on  a  sample 
grid  of  size  512  x  512.  Since  each  of  the  transitions  from  a  disk  to  the  background  and  vice 
versa  generates  frequencies  ranging  from  —  °o  to  +°o ,  the  representation  on  a  512x512  grid 
of  samples  corresponds  to  undersampling.  The  aliasing  of  the  spectral  components  can  be  seen 
in  Fig.  27b,  which  displays  the  magnitude  spectrum  of  this  image.  One  may  also  note  that 
while  the  aliasing  is  rather  small  and  unnoticeable  (since  it  is  negligible  compared  to  the 
numerical  precision  errors)  at  the  center  of  the  spectrum  (low  frequency  range),  some 
noticeable  aliasing  is  present  at  the  extremities  (large  frequencies). 
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Figs.  27a  and  27b.  Object  represented  on  a  512  x  512  grid  and  its  spectrum 


Let  us  repeat  the  blurring  operation  earlier  described  in  Chapter  3  on  this  object.  Fig. 
28a  shows  the  blurred  image  resulting  from  the  convolution  of  the  image  in  Fig.  27a  with  an 
OTF  of  a  sensor  with  a  circular  aperture  of  diameter  16  pixels.  Since  no  frequency  components 
beyond  the  cutoff  frequency  of  16  are  present  in  the  blurred  image,  a  representation  of  this 
image  on  a  grid  of  512x512  samples  constitutes  oversampling  (sampling  rate  exceeding  the 
Nyquist  rate).  Hence  no  aliasing  of  spectral  components  will  be  present  which  is  clearly  seen 
in  Fig.  28b.  Fig.  29a  shows  a  Nyquist  sampled  version  of  the  blurred  image  which  is  a 
representation  on  a  grid  of  32 x  32  samples.  Since  the  sampling  rate  is  just  adequate  to  prevent 
an  overlap  of  spectral  components,  no  aliasing  results  which  can  be  seen  in  Fig.  29b.  It  is  to  be 
noted  that  the  5 12  x  512  image  in  Fig.  28a  is  a  8x  oversampled  version  of  the  Nyquist  sampled 
64x64  image  in  Fig.  29a.  Due  to  the  preservation  of  integrity  of  spectral  components,  both 
images  contain  the  same  amount  of  information;  indeed,  one  can  be  transformed  into  the  other 
without  any  loss  of  information  through  “resampling”  operations  that  will  be  discussed  in  a 
later  section. 


Figs.  28a  and  28b.  Blurred  image  and  its  spectrum 


Figs.  29a  and  29b.  Blurred  image  represented  on  a  32 x  32  grid  and  its  spectrum 
5.2  Need  for  Oversampling  in  Super-resolution  Processing 

While  sampling  at  the  Nyquist  rate  by  a  selection  of  the  sampling  pitch 
px  =  py  =  k  /  coc  corresponds  to  an  optimal  spacing  of  detector  elements  or  a  scan  rate  that 

provides  the  largest  dwell  time  (thus  enabling  an  increased  SNR  in  the  acquired  image),  it  will 
lead  to  some  problems  when  the  Nyquist  sampled  image  is  subjected  to  super-resolution 
processing.  If  the  super-resolution  algorithm  employed  successfully  extrapolates  the  image 
bandwidth  beyond  the  cutoff  frequency  coc ,  say  up  to  an  extended  frequency  limit 
coe(a)e  >  coc) ,  due  to  the  periodic  replication  of  spectral  components  the  new  frequencies 
created  in  the  frequency  range  coc<co  <coe  will  overlap  with  the  original  frequency 

components  present  in  Gs(jQx,jCl^ .  Not  only  will  the  new  frequencies  created  become 

inaccurate  but  the  original  frequency  components  present  in  the  passband  of  the  image  become 
corrupted  due  to  the  aliasing,  which  now  extends  over  the  entire  frequency  range 

2 coc-coe  <  co  <  Q)e .  This  in  turn  will  produce  aliasing  artifacts  in  the  restored  image  f(x,y ) 

which  is  the  output  of  super-resolution  processing.  Due  to  the  corruption  of  the  passband,  the 
restored  image  may  turn  out  to  be  poorer  than  the  original  image. 


In  order  to  prevent  aliasing,  the  output  f(x,y)  must  be  represented  on  a  grid  of 

samples  finer  than  the  Nyquist  sampled  input  image  g(m,n),  which  in  turn  requires  a  higher 
sampling  rate  to  be  used  for  the  representation  of  the  sampled  image  (thus  increasing  the  image 
size).  To  determine  the  amount  of  oversampling  required,  one  may  anticipate  the  width  of 
frequency  extension  that  may  reliably  be  expected  from  the  super-resolution  algorithm 
employed.  Thus,  if  a  4x  improvement  in  resolution  is  anticipated,  one  would  expect  to  see  the 
creation  of  object  frequencies  up  to  approximately  coe  =  4 coc .  Hence  to  ensure  availability  of 
support  needed  for  this  bandwidth  extension,  the  image  to  be  processed  will  need  to  be 
represented  on  a  finer  grid  of  samples  corresponding  to  4  times  the  Nyquist  rate.  This 
requirement  in  turn  implies  the  selection  of  a  smaller  pitch  px  =  py  =  n  /  Acoc.  The  four-fold 

increase  in  the  image  size  needs  to  be  noted  in  this  case. 

Once  the  object  estimate  f(x,y )  is  generated  as  the  output  of  super-resolution 

processing,  it  can  be  represented  on  the  original  coarser  grid  of  samples  corresponding  to  the 
Nyquist  sampling  rate  by  a  downsampling  operation.  This  step  merely  involves  an  aggregation 
of  samples  and  requires  simple  averaging  calculations.  The  need  for  downsampling  or  the 
desire  to  represent  the  constructed  object  estimate  on  a  coarser  grid  of  samples  may  arise 
simply  from  the  efficiency  it  provides  for  storing  and  transmitting  this  data  with  reduced 
communication  overhead.  In  several  applications,  such  as  in  a  multispectral  missile  seeker  that 
employs  a  number  of  diverse  sensors,  reduction  of  communication  overhead  is  of  paramount 
importance  for  effective  utilization  of  processed  data  (  for  illustration,  for  effective  fusion  of 
data  coming  from  a  number  of  sensor  channels  continuously  into  the  fusion  processor)  [37]. 

5.3  Hardware  and  Software  methods  for  Obtaining  Oversampled  data 

Obtaining  sampled  image  data  at  a  rate  higher  than  the  Nyquist  rate  can  be 
accomplished  during  data  acquisition  by  modifying  the  hardware  or  as  a  post-acquisition  signal 
processing  step.  In  the  latter  case,  additional  software  to  perform  signal  resampling  [4,38]  may 
need  to  be  included  as  part  of  the  overall  processing.  PMMW  sensor  platforms  currently  being 
developed  have  attempted  to  include  timing  circuitry  that  provides  the  capability  for  acquiring 
oversampled  imagery  data  [27,39].  While  some  recent  demonstrations  have  discussed  the 
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capability  of  collecting  oversampled  data  (at  a  rate  as  high  as  16  times  the  Nyquist  rate),  there 
are  some  significant  tradeoffs  this  may  entail,  which  in  turn  may  render  software  methods 
considerably  more  attractive  for  accomplishing  the  desired  goals.  At  the  outset  it  must  be  noted 
that  acquisition  of  oversampled  data  will  typically  require  significant  hardware  modifications. 
One  is  forced  to  use  either  a  larger  focal  plane  array  with  more  detector  elements  packed 
closely  together,  which  incurs  a  higher  cost,  or  attempt  to  include  a  precision  scan  mechanism 
which  permits  smaller  scan  steps  through  electronic  scanning  methods.  In  the  latter  case,  it  is 
important  to  note  that  scan  speed  is  closely  related  to  the  integration  time  of  the  sensor  and 
consequently  has  a  major  effect  on  the  thermal  sensitivity.  Considering  the  fact  that  at  the 
present  state  of  development  of  PMMW  imaging  technology,  the  major  problem  is  the  slow 
response  due  to  poor  thermal  sensitivity,  increasing  the  scan  speed  may  impose  serious 
limitations  on  the  quality  of  data  (available  SNR  in  the  acquired  image).  Yet  another  method 
that  is  being  investigated  is  the  use  of  microscanning  [40]  which  combines  a  number  of  slightly 
translated  images  into  a  single  larger  size  image.  Several  techniques  have  been  proposed  to 
obtain  microscanned  PMMW  images;  all  of  these  involve  introduction  of  a  subpixel  dithering 
applied  either  to  the  PMMW  antenna  or  to  the  focal  plane  array.  Once  again,  the  reduction  in 
the  frame  rate  and  the  integration  time  will  pose  serious  limitations  in  this  case  as  well.  It  must 
be  noted  in  passing  that  microscanning  is  not  necessarily  used  for  oversampling  always  and 
even  to  obtain  a  sampled  image  at  the  Nyquist  rate  one  may  need  to  employ  microscanning 
methods,  As  Bradley  and  Dennis  [40]  point  out,  focal  plane  arrays  cannot  be  packed  together 
closely  enough  to  sample  all  of  the  spatial  frequencies  that  they  are  able  to  detect  and  hence 
microscanning  by  dithering  the  images  by  half  a  detector  pitch  may  be  needed  to  obtain 
sampled  data  at  the  Nyquist  rate. 

In  contrast  to  the  above  methods,  signal  processing  methods  provide  a  number  of 
benefits  for  resampling  of  the  image  data.  Resampling  operations  permit  changing  the  number 
of  pixels  that  define  a  sampled  image  in  a  new  representation.  If  the  total  number  of  pixels  is 
increased  compared  to  the  original  grid,  the  operation  is  called  “upsampling”,  while  a 
downward  revision  of  the  total  number  of  pixels  leading  to  a  coarser  sample  grid  is  called 
“downsampling”.  Upsampling  typically  involves  interpolation  of  available  data  while 
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downsampling  involves  an  averaging  of  data  available.  Introduction  of  upsampling  operations 
can  provide  oversampled  image  data  that  can  be  readily  used  as  input  to  super-resolution 
processing  for  avoiding  aliasing  artifacts.  A  considerable  flexibility  exists  in  upsampling  a 
given  sampled  data  set.  For  application  to  image  data,  a  number  of  possible  approaches  exist 
and  upsampling  can  be  performed  either  on  the  space  domain  data  directly  or  in  the  frequency 
domain  using  Fourier  transform  operations.  The  tradeoffs  between  processing  complexity  and 
the  quality  of  the  interpolated  values  need  to  be  considered  in  selecting  a  specific  approach  for 
a  given  application. 

Methods  that  employ  space-domain  operations  for  interpolation  are  generally  less 
complex  to  implement.  The  simplest  method  is  the  zero-order  hold,  also  called  nearest 
neighbor  assignment  [38],  which  involves  assigning  to  the  interpolated  element  the  value  of  the 
nearest  known  pixel.  This  is  the  fastest  to  implement  of  all  the  approaches.  The  next  in  the 
degree  of  complexity  is  the  first-order  hold,  also  called  bilinear  sampling  [38],  which  uses  the 
four  closest  known  pixels  that  surround  a  new  point  to  linearly  interpolate  its  value.  Evidently, 
the  zero-order  and  first-order  holds  give  piecewise  and  linear  approximations  respectively 
between  the  samples.  Higher  order  approximations  such  as  second-order  and  third-order  holds 
give  quadratic  and  cubic  spline  approximations.  As  the  order  of  the  hold  approaches  infinity, 
the  interpolating  function  tends  to  a  properly  scaled  Gaussian  function  [4] .  It  may  also  be  noted 
that  as  the  order  of  the  hold  increases,  the  interpolation  error  decreases  (leading  to  better 
quality  data),  but  the  resolution  loss  will  increase. 

Frequency-domain  methods  for  upsampling  offer  more  attractive  alternatives  in  terms 
of  the  quality  of  interpolated  data.  For  a  sampled  image  g(m,n)  of  size  MxN ,  the  Discrete 
Fourier  Transform  (DFT)  coefficients  that  describe  the  image  in  the  frequency  domain  are 
given  by 
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In  order  to  upsample  this  image  to  produce  a  new  image  of  size  MxN,  where 
M  >  M,N  >  N,  the  image  spectrum  is  zero  padded  to  the  desired  size  by  defining  the  DFT 
coefficients  in  the  form 
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It  is  to  be  noted  that  the  zero-padded  spectrum  is  of  dimension  MxN .  The  inverse  DFT  of 
G(k,l)  can  now  be  calculated  to  yield  the  new  image  of  size  MxN  by 
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It  can  be  shown  that  the  zero  padding  in  the  frequency-domain  as  described  above  is  equivalent 
to  performing  interpolation  in  the  space-domain  with  a  sine  function  (sinx/x).  Furthermore, 
since  all  of  the  spectrum  of  the  original  image  is  retained  in  the  new  image,  this  interpolation 
does  not  introduce  any  approximation  errors  as  in  the  case  of  space  domain  interpolation 
methods,  excepting  for  the  numerical  errors  related  to  the  use  of  DFT  and  inverse  DFT 
operations. 


An  illustration  of  the  performance  of  the  frequency-domain  interpolation  method 
discussed  above  can  be  given  by  an  16x  upsampling  of  the  32  x  32  image  shown  in  Fig.  29a. 
The  upsampled  512  x  512  image  is  shown  in  Fig.  30a  and  the  corresponding  spectrum  is  shown 
in  Fig.  30b.  Comparing  these  to  the  image  in  Fig.  28a  and  its  spectrum  shown  in  Fig.  28b,  one 
may  note  they  are  almost  identical.  The  images  are  not  identical  in  pixel  values,  however,  due 
to  the  numerical  errors  introduced  by  DFT  operation. 
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Fig.  30a  and  30b.  16x  upsampled  version  of  image  in  Fig.  29a  and  its  spectrum 

Downsampling  is  the  reverse  of  upsampling  and  produces  an  image  of  a  reduced  size. 
As  in  the  case  of  upsampling,  a  given  image  can  be  downsampled  by  operations  executed  in  the 
space-domain  or  in  the  frequency-domain.  In  the  space-domain  it  involves  averaging  the  pixel 
values,  whereas  in  the  frequency-domain  one  would  reduce  the  spectrum  to  the  desired  reduced 
size  by  extracting  the  portion  at  the  center  of  the  spectrum  of  the  original  image.  In  particular, 
it  must  be  noted  that  the  frequency-domain  upsampling  does  not  eliminate  any  information 
present  in  the  original  image  and  hence  it  is  always  possible  to  recover  the  original  image  from 
its  upsampled  version.  This  however  may  not  be  true  with  the  downsampling  operation  on  a 
given  image  since  this  operation  can  throw  away  some  information  which  may  be  impossible  to 
recover  from  the  downsampled  version.  It  may  also  be  noted  that  due  to  the  use  of  DFT  in 
going  back  and  forth  between  the  space-  and  frequency-domains,  the  upsampling  and 
downsampling  operations  can  increase  or  decrease  the  number  of  points  used  to  describe  the 
image  only  by  powers  of  two. 

How  does  the  upsampling  operation  compare  with  hardware  methods  in  obtaining 
oversampled  data?  Being  a  signal  processing  operation  executed  in  software,  upsampling  is 
much  simpler  to  implement  and  eliminates  the  costly  modifications  needed  for  hardware 
oversampling.  In  regard  to  the  quality  of  data,  while  space-domain  interpolation  methods  can 
introduce  approximation  errors  (depending  on  the  order  of  approximation),  frequency-domain 
upsampling  will  not  lead  to  any  errors  (other  than  the  numerical  errors  in  implementation  of 


DFT  operations,  as  noted  earlier).  In  fact,  it  can  be  shown  that  for  an  image  that  is  Nyquist 
sampled,  under  zero  noise  conditions  the  frequency-domain  upsampling  operation  will  provide 
an  enlarged  image  that  is  comparable  in  accuracy  to  an  image  that  is  acquired  by  a  sensor  with 
a  higher  density  of  detector  elements.  This  property  together  with  the  ease  of  implementation 
makes  upsampling  a  more  attractive  alternative  to  hardware  methods  of  oversampling  which 
have  the  disadvantages  of  additional  complexity  and  cost,  reduced  frame  rate,  and  reduced 
integration  time  (leading  to  poor  SNR  in  the  acquired  images). 

5.4  Selection  of  Needed  Degree  of  Oversampling 

To  obtain  an  understanding  of  what  an  oversampled  version  of  data  will  do  to  the 
restoration  and  super-resolution  process,  an  experiment  was  conducted  by  implementing  the 
ML  restoration  algorithm  given  by  Equation  (24)  on  the  blurred  image  shown  in  Fig.  29a.  The 
Nyquist-sampled  image  of  size  32  x  32  was  processed  by  50  iterations  of  the  ML  restoration 
algorithm.  The  resultant  is  also  a  32x  32  image.  However,  for  a  direct  comparison,  the 
processed  image  is  also  upsampled  for  representation  on  a  512x512  grid,  which  is  shown  in 
Fig.  31a.  Comparing  the  images  in  Figs.  30a  and  31a,  any  improvements  are  hardly 
noticeable.  A  further  confirmation  of  this  is  also  obtained  by  examining  the  spectrum  of  the 
processed  image  which  is  shown  in  Fig.  31b  represented  on  a  512x512  grid.  It  is  clear  that 
not  much  super-resolution  has  resulted  from  the  restoration  processing,  which  is  explainable 
from  the  lack  of  support  available  for  the  creation  of  new  frequencies  in  this  case.  However, 
what  seems  to  be  more  surprising  is  that  not  much  of  restoration  seems  to  have  taken  place 
either  due  to  the  strong  artifacts  generated  from  processing. 

In  order  to  provide  the  needed  support  for  the  possible  generation  of  new  frequencies, 
upsampled  versions  of  the  blurred  image  were  subjected  to  processing  with  the  same  number  of 
iterations  (  specifically  50).  To  examine  any  differences  in  the  achievable  restoration,  different 
degrees  of  oversampling  (  2x,  4x,  8x,  and  16x  resulting  in  images  of  sizes  64  x  64,  128  x  128, 
256  x  256,  and  512x  512,  respectively  )  were  used.  Fig.  32a  shows  the  result  of  processing  the 
16x  upsampled  image  which  is  displayed  on  the  original  grid  size  of  512  x  512  .  The  spectrum 
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of  this  image  is  shown  in  Fig.  32b.  The  expansion  of  the  spectrum  as  well  as  the  de-blurring 
effects  are  clearly  visible  in  these  figures. 


Figs.  31a  and  31b.  Result  of  processing  the  Nyquist-sampled  (64x64)  image  and  spectrum 


Figs.  32a  and  32b.  Result  of  processing  16x  upsampled  (512x512)  image  and  its  spectrum 

In  order  to  obtain  a  greater  insight  into  the  restoration  performance  observed  in  this 
experiment,  the  convergence  behavior  of  the  restoration  error  as  well  as  the  rate  of  increase  in 
the  likelihood  function  were  examined.  Since  we  are  starting  with  a  known  object  in  this 
experiment  (  i.e.,  image  in  Fig.  27a),  one  can  easily  compute  the  L,  -norm  of  the  deviation 
between  the  object  and  the  processed  image  as  iterations  progress.  Furthermore  at  each 
successive  iteration,  the  value  of  the  likelihood  function  can  be  computed  from  noting  that  the 
logarithm  of  the  likelihood  function  is  given  by 
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Figs.  33  and  34  display  the  convergence  of  the  error  and  the  increase  in  likelihood  as 
iterations  progress  in  the  various  cases.  In  Fig.  33,  the  top  curve  shows  the  decay  of  error  in 
processing  the  Nyquist-sampled  image,  the  middle  curve  when  processing  the  2x  upsampled 
image,  and  the  bottommost  curve  when  processing  the  4x,  8x  and  16x  upsampled  images.  It  is 
of  interest  to  note  that  while  the  processing  of  upsampled  versions  of  the  blurred  image  results 
in  reduced  restoration  errors  (which  agrees  with  the  explanation  given  above),  no  appreciable 
difference  exists  in  the  three  cases  of  4x,  8x  and  16x  oversampling  although  the  images  in  the 
latter  cases  are  considerably  bigger  than  the  former  with  corresponding  increases  in 
computational  complexity.  A  farther  confirmation  of  this  behavior  is  also  evident  from  Fig.  34 
where  the  lower  curve  shows  the  rate  of  likelihood  increase  for  the  Nyquist-sampled  case  while 
the  upper  curve  shows  the  increase  for  the  2x,  4x,  8x  and  16x  oversampled  cases.  Clearly, 
there  are  no  noticeable  differences  in  the  restoration  of  the  2x,  4x,  8x  and  16x  oversampled 
images. 


Fig.  33.  Convergence  of  restoration  error 
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Fig.  34.  Likelihood  increase  with  progress  of  iterations 

The  last  observation  apparently  has  some  important  implications  to  several  efforts 
currently  being  mounted  for  collection  of  oversampled  data  by  implementing  modifications  to 
data  acquisition  circuitry,  specially  for  sensing  PMMW  imagery  data.  Some  recent  studies 
[27,42]  have  proposed  using  microscanning  techniques  for  acquiring  PMMW  data  at  16x 
oversampled  rate  in  order  to  achieve  greater  resolution  gains  through  super-resolution 
processing.  Our  results  evidently  do  not  support  such  a  proposition.  For  the  simulation 
experiment  discussed  earlier,  there  is  little  need  for  oversampling  beyond  the  2x  rate.  Besides 
the  need  to  process  larger  sized  images,  which  increases  the  computational  burden  with  little  or 
no  return  in  restoration  performance,  collection  of  oversampled  data,  either  by  increasing  the 
density  of  detector  elements  in  the  sensor  array  or  by  microscanning  techniques,  will  have 
detrimental  effects  on  the  image  SNR  due  to  the  reduced  integration  times. 

5.5  Design  of  a  Progressive  Upsampling  Scheme  for  Iterative  Restoration 

From  the  discussion  given  in  the  earlier  sections  two  important  factors  stand  out.  First, 
processing  of  an  oversampled  image  is  necessary  in  order  to  ensure  any  possible  super¬ 
resolution  from  the  iterative  algorithm;  however,  an  arbitrarily  large  degree  of  oversampling 
may  not  yield  any  benefits  but  may  unnecessarily  increase  the  computational  burden.  Second, 
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while  the  resolution  improvements  could  be  substantial  during  the  first  few  iterations,  the  later 
iterations  may  not  provide  corresponding  benefits  and  may  in  fact  be  not  cost  effective 
(considering  the  computational  effort  needed  for  implementing  each  iteration). 

The  two  factors  cited  above  suggest  a  more  intelligent  way  of  implementing  the 
iterative  algorithm  by  progressively  increasing  the  rate  of  upsampling  in  order  to  provide  the 
range  of  support  required  for  frequency  extension  only  as  needed.  Thus,  one  may  commence 
the  restoration  of  the  Nyquist  sampled  blurred  image  and  after  a  few  iterations  upsample  by  2x 
to  perform  the  next  cycle  of  iterations.  The  rationale  for  working  with  the  Nyquist  sampled 
image  at  this  step  is  that  most  of  the  improvement  in  the  restored  image  comes  from  the 
passband  restoration  ( i.e.  for  reversing  the  effects  of  convolution  within  the  passband  of  the 
image)  and  hence  the  processing  of  the  smallest  sized  image  is  useful  at  this  step.  The  2x 
upsampled  image  will  have  a  size  four  times  that  of  the  original  image  but  will  also  provide  a 
frequency  range  coc  <  co  <  cos  /  2  for  the  creation  of  new  frequencies  so  that  they  will  not 
overlap  any  frequencies  within  the  image  passband,  where  8>t  is  the  new  sampling  frequency 
corresponding  to  the  2x  upsampled  version  of  the  image  (specifically,  cos  =  2cos,  where 
os=2coc  for  the  Nyquist  sampled  representation).  Extrapolation  of  the  object  spatial 
frequencies,  i.e.  super-resolution,  will  begin  at  this  step  and  can  be  continued  until  an  iteration 
step  where  significant  benefits  from  processing  are  no  longer  noticeable.  The  degree  of 
upsampling  can  now  be  increased,  say  to  4x  of  the  original  image,  and  the  iterations  continued. 
A  schematic  that  describes  this  scheme  of  implementing  m  iterations  of  the  ML  restoration 
algorithm  followed  by  a  2x  upsampling  operation  is  shown  in  Fig.  35. 


107 


Fig.  35.  The  Progressive  Upsampling  Scheme  for  Implementation  of  ML  Algorithm 


For  illustrating  the  performance  of  the  progressive  upsampling  scheme  described  above, 
an  experiment  was  conducted  by  performing  ML  restoration  of  the  blurred  “concentric  disks” 
image  that  was  considered  earlier.  Starting  with  a  Nyquist-sampled  representation  of  the 
blurred  image  of  size  32  x  32,  a  first  cycle  of  five  ML  iterations  were  executed  and  the  result 
was  upsampled  to  a  64  x  64  image.  This  image,  which  is  a  representation  on  a  finer  grid,  was 
then  subjected  to  the  next  cycle  of  five  ML  iterations.  The  result  was  upsampled  to  a  128  x  128 
image  and  was  then  processed  with  the  next  cycle  of  five  ML  iterations  after  which  it  was 
upsampled  again  to  a  256  x  256  image  and  subjected  to  the  next  cycle  of  five  ML  iterations. 
The  result  was  again  upsampled  to  yield  a  512  x512  image  which  was  then  processed 
iteratively  for  the  next  30  iterations.  The  processed  image  at  the  end  of  this  experiment  is 
shown  in  Fig.  36a  and  its  spectrum  is  shown  in  Fig.  36b. 


Figs.  36a  and  36b.  Result  of  processing  with  progressive  upsampling  scheme  and  its  spectrum. 
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A  plot  of  the  convergence  of  the  4- norm  of  the  deviation  between  the  object  and  the 
processed  image  in  this  case  as  iterations  progress  is  shown  in  Fig.  37.  For  comparison,  a 
convergence  plot  of  the  4 -norm  in  the  case  of  processing  the  Nyquist-sampled  version  of  the 
blurred  image  from  the  start  of  the  iterations  is  also  given  in  the  same  figure.  It  is  of  interest  to 
note  that  the  curve  breaks  downwards  sharply  at  the  point  of  upsampling  due  to  the 
representation  of  the  problem  on  a  finer  grid  and  the  restart  of  a  new  iteration  cycle,  which 
exploits  the  property  of  faster  convergence  at  the  commencement  of  iterations  as  noted  before. 
A  second  downward  break,  not  as  noticeable  as  the  first  one,  also  takes  place  at  the  next 
upsampling  instant.  The  net  effect  of  the  progressive  upsampling  operation  can  hence  be  seen 
as  a  faster  restoration. 


Fig.  37.  Convergence  of  Restoration  Error  with  Progressive  Upsampling 

A  comparison  of  the  spectrum  in  Fig.  36b  with  that  of  the  blurred  image  in  Fig.  28b 
gives  a  clear  indication  of  the  expanded  frequency  content  and  hence  the  super-resolution 
performance  of  the  algorithm.  Also,  comparing  the  spectrum  of  the  processed  image  with  that 
in  Fig.  32b,  which  displays  the  results  of  processing  an  16x  oversampled  image  (  giving  an 
image  size  of  512  x  512)  for  50  iterations,  it  is  clear  that  a  comparable  level  of  super-resolution 
performance  is  obtained  with  far  less  number  of  iterations.  The  increased  computational 
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efficiency  resulting  from  the  need  to  process  only  images  of  smaller  sizes  initially  and  growing 
the  size  only  as  needed  deserves  to  be  noted. 

The  processing  step  at  which  to  perform  upsampling  is  largely  a  subjective  decision.  A 
guideline  that  can  be  used  in  the  identification  of  an  appropriate  instant  for  upsampling  is 
when  the  rate  of  decrease  in  the  error  (4 -norm)  has  reduced  to  less  significant  values  (i.e.  the 
convergence  has  reached  a  relative  plateau).  While  this  evaluation  based  on  the  restoration 
error  is  possible  when  the  object  that  resulted  in  the  blurred  image  is  known  (as  is  the  case  in 
the  present  example),  in  practical  image  restoration  tasks  this  is  not  available.  In  these  cases 
however,  one  could  examine  the  behavior  of  the  likelihood  function  (given  by  Equation  (87) 
with  /  replaced  by  the  estimated  signal  /)  and  base  the  decision  on  performing  the 
upsampling  at  the  points  when  the  rate  of  increase  in  the  likelihood  function  has  reduced  to  a 
value  less  than  what  is  desired  based  on  the  initial  rate  of  increase  achieved.  An  examination  of 
the  likelihood  function  can  also  be  used  to  terminate  the  ML  iterations  at  a  processing  step  that 
yields  the  maximum  value  of  the  likelihood.  A  comparison  of  the  increase  in  the  likelihood 
function  between  the  processing  of  the  Nyquist  sampled  image  for  all  50  iterations  without 
upsampling  and  with  progressive  upsampling  is  shown  in  Fig.  38. 


Fig.  38.  Comparison  of  likelihood  increase 
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In  order  to  further  validate  the  performance  benefits  resulting  from  the  ML  restoration 
algorithm  aided  with  progressive  upsampling,  several  restoration  experiments  on  degraded 
images  were  conducted.  In  this  section,  we  shall  briefly  outline  the  result  from  one  of  these 
experiments. 

The  super-resolution  performance  of  the  algorithm  was  tested  by  processing  a  256  x  256 
image  from  our  database  (“Lenna”  image).  Fig.  39a  shows  the  original  image  used  in  this 
experiment.  Fig.  39b  shows  the  blurred  image  obtained  by  convolution  with  the  PSF  of  a 
sensor  with  cutoff  o)c  =  16.  The  spectrum  of  the  blurred  image  is  shown  in  Fig. 39c.  For 
performing  restoration,  the  blurred  image  is  upsampled  to  a  512  x  512  image  (  with  16x 
oversampling)  and  subjected  to  80  iterations  of  ML  algorithm.  The  result  of  this  processing  is 
shown  in  Fig.  39d.  The  blurred  image  was  also  restored  with  progressive  upsampling  by 
commencing  the  iterative  processing  with  an  original  32  x  32  image  (corresponding  to  a 
Nyquist  sampling  of  the  blurred  image)  and  upsampling  by  2x  after  executing  every  5 
iterations  of  ML  algorithm  until  the  final  image  size  of  512  x  512  was  obtained.  The 
processing  was  terminated  at  the  completion  of  the  40th  ML  iteration  from  the  start.  The  result 
of  this  processing  is  shown  in  Fig.  39e  and  the  corresponding  spectrum  is  shown  in  Fig.  39f.  A 
comparison  of  the  convergence  performance  in  the  two  implementations  is  shown  in  Fig.  40, 
which  displays  a  more  attractive  convergence  behavior  with  the  progressive  upsampling 
scheme.  In  particular  it  is  of  interest  to  note  that  the  error  value  with  progressive  upsampling 
at  the  40th  iteration  is  smaller  than  the  error  at  the  80th  iteration  for  processing  the  16x 
oversampled  image,  which  is  obtained  in  addition  to  a  significant  saving  in  computational 

burden. 


ill 


Fig.  39d  Fig.39e  Fig.  39f 

Fig.  39.  Results  of  Processing  “Lena”  Image 
Fig.  39a.  Original  Image;  Fig.  39b.  Blurred  Image;  Fig.  39c.  Spectrum  of  Blurred  Image; 

Fig.  39d.  Restored  Image  with  no  Upsampling;  Fig.  39e.  Restored  with  Progressive 
Upsampling;  Fig.  39f.  Spectrum  of  Restored  Image  by  Progressive  Upsampling; 


Series  1 :  80  iterations  without  upsampling 

Series  2:  40  iterations  with  upsampling  (every  5  iterations,  32x32  to  512x512) 
Fig.  40.  Comparison  of  Convergence  Performance  of  the  Two  Implementations. 
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6.  CONCLUSIONS 


6.1  Summary  of  Results  Obtained 

This  project  was  primarily  aimed  at  the  design  and  development  of  novel  super¬ 
resolution  algorithms  for  processing  Passive  Millimeter-wave  (PMMW)  imagery  data.  The 
principal  objective  of  this  effort  was  to  evaluate  and  recommend  specific  techniques  for  super¬ 
resolution  that  will  enable  the  U.S.  Air  Force  to  significantly  enhance  the  usefulness  of 
PMMW  images  it  obtains  for  military  intelligence  purposes.  Several  investigations  were 
conducted  towards  this  objective  results  of  which  are  described  in  this  report. 

During  the  current  project,  our  investigations  mainly  focused  on  the  design  of  iterative 
super-resolution  algorithms  that  can  be  developed  using  a  statistical  modeling  of  the  image 
formation  process.  The  basic  idea  underlying  these  methods  is  to  account  for  the  statistical 
behavior  of  emitted  radiation  at  the  level  of  individual  photon  events  by  constructing 
appropriate  object  radiance  distribution  models  (using  knowledge  of  fluctuation  statistics). 
Employing  this  mathematical  framework  enabled  formulating  the  image  restoration  problem  as 
mathematical  optimization  problems  that  yield  Maximum  Likelihood  (ML)  or  Maximum  A 
Posteriori  (MAP)  estimates  of  the  restored  object.  Of  particular  interest  to  our  work  in  this 
project,  this  framework  also  offered  the  feasibility  of  tailoring  super-resolution  algorithms 
whose  spectrum  extrapolation  performance  can  be  monitored  and  quantitatively  measured.  As 
established  from  previous  work,  two  major  factors  that  could  limit  successful  implementations 
of  image  restoration  and  super-resolution  algorithms  in  practice,  particularly  in  missile  seeker 
applications,  are: 

(1)  lack  of  accurate  knowledge  of  sensor  point  spread  function  (PSF)  parameters, 
and 

(2)  noise-induced  artifacts  in  the  restoration  process. 

Our  investigations  hence  addressed  obtaining  a  characterization  of  these  factors  and  focused  on 
the  design  of  restoration  algorithms  that  possess  a  certain  degree  of  robustness  to  inaccuracies 
in  the  knowledge  of  PSF  parameters  and  also  can  suppress  the  noise-induced  artifacts. 
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In  order  to  facilitate  performing  restoration  studies  on  PMMW  imagery  data,  some 
efforts  were  also  directed  to  building  up  a  PMMW  image  database,  and  we  now  have  a  number 
of  images  acquired  from  several  state-of-the-art  PMMW  radiometers  developed  by  different 
hardware  groups,  specifically  Air  Force  Research  Labs  at  Eglin  AFB,  Army  Research  Lab  at 
Adelphi,  MD,  and  TRW,  Inc.  at  Redondo  Beach,  CA.  The  studies  that  were  conducted 
included  a  demonstration  of  resolution  enhancements  by  processing  various  PMMW  images  in 
our  database  with  these  algorithms. 

Several  useful  results  have  been  obtained  through  our  work  on  this  project.  Some  of  the 
more  interesting  ones  will  be  briefly  listed  here: 

•  An  analytical  description  of  image  restoration  and  super-resolution  objectives  from 
a  spectrum  reconstruction  viewpoint  has  been  developed  which  facilitates 
mathematically  formulating  the  restoration  and  super-resolution  problems  in  an 
optimization  framework. 

•  Since  satisfactory  restoration  and  super-resolution  of  PMMW  images  can  be 
achieved  only  through  an  intelligent  tailoring  of  iterative  processing  algorithms,  an 
analysis  of  some  fundamental  issues  that  need  to  be  taken  into  account  in 
synthesizing  algorithms  that  afford  simple  digital  implementation  were  conducted 
and  several  useful  conclusions  were  developed.  In  particular,  the  important  role 
played  by  the  selection  of  a  sampling  grid  size  on  the  overall  resolution  achievable 
through  digital  processing  of  image  data  was  examined  in  detail,  which  led  to  the 
formulation  of  a  progressive  upsampling  procedure  for  optimized  implementations 
of  restoration  and  super-resolution  algorithms. 

•  A  database  of  PMMW  imagery  data  collected  using  a  number  of  different 
radiometer  designs  by  different  groups  (AF  Wright  Lab  Armament  Directorate, 
TRW  Inc.,  and  Army  Research  Lab)  has  been  developed.  This  database  will  be  very 
valuable  for  conducting  any  future  studies  on  the  characterization  and  restoration  of 
PMMW  images. 

•  Using  several  PMMW  images  in  this  database,  specific  experiments  are  conducted 
to  demonstrate  the  resolution  enhancement  and  spectrum  extrapolation  resulting 
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from  an  iterative  restoration  algorithm  developed  using  a  Maximum  Likelihood 
(ML)  framework. 

•  A  processor  requirements  analysis  is  conducted  for  evaluating  the  practical 
implementation  of  the  iterative  ML  algorithm  using  commercial  microprocessors  in 
order  to  demonstrate  that  available  chips  such  as  TMS320C6x  can  support 
implementation  of  this  algorithm  while  satisfying  desired  frame-rate  requirements. 

•  Since  image  restoration  involves  essentially  an  inversion  operation,  the  robustness 
of  restoration  algorithms  to  variations  in  the  estimate  of  the  sensor  point  spread 
function  (PSF)  is  of  particular  importance.  Several  experiments  are  conducted  to 
establish  the  robustness  properties  of  the  iterative  ML  super-resolution  algorithm. 

•  For  obtaining  an  iterative  correction  of  the  sensor  PSF  from  a  starting  estimate,  a 
blind  deconvolution  algorithm  that  jointly  performs  image  restoration  and  PSF 
estimation  is  developed  and  the  convergence  properties  of  this  algorithm  are 
evaluated. 

•  Since  the  presence  of  significant  amounts  of  noise  in  the  measured  data  (poor  SNR 
levels)  is  a  source  of  generation  of  noise-induced  restoration  artifacts,  a  study  of 
the  dynamics  of  information  transfer  between  the  noise  present  and  the  image  being 
restored  during  successive  iterations  of  restoration  processing  is  of  particular 
importance.  By  conducting  quantitative  performance  evaluations,  the  effects  of 
noise  on  the  processed  image  as  well  as  on  the  accuracy  of  PSF  estimation  are 
examined. 

•  A  modified  ML  algorithm  that  combines  a  post-filtering  PSF  adjustment  step  with 
the  restoration  processing  is  developed  and  its  capabilities  for  suppressing  noise- 
induced  artifacts  are  established. 

•  Since  the  degree  of  spectrum  extrapolation  achievable  from  super-resolution 
processing  depends  on  the  spatial  extent  of  objects  in  the  scene,  a  procedure  for 
separation  of  image  background  from  the  details  is  developed.  This  background- 
detail  separation  procedure  facilitates  subjecting  only  the  detail  portion  of  the  image 
to  super-resolution  processing  which  in  turn  results  in  superior  restoration  of 
images. 
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•  The  problem  of  assessing  the  improvement  in  image  quality  subsequent  to 
restoration  processing  is  studied  and  various  metrics  in  spatial  and  spectral  domains 
are  developed. 

•  Since  the  goal  of  super-resolution  processing  is  to  obtain  a  processed  image  that  is 
equivalent  to  one  obtained  from  a  larger  aperture  (higher  cost)  sensor,  a  new  metric 
that  measures  the  “virtual  aperture  size”  of  the  sensor  is  developed  for  quantifying 
the  resolution  improvement  achieved  from  a  restoration  algorithm. 

6.2  Some  Directions  for  Further  Research 

Results  obtained  from  this  study  can  be  improved  by  further  investigations  along 
several  directions.  One  of  the  most  promising  directions  seems  to  be  the  utilization  of  set 
theory-based  estimation  methods  and  integrating  these  with  the  statistical  optimization 
approaches  used  in  the  present  study.  From  our  work  on  this  project,  as  well  as  prior  studies 
by  others,  it  is  now  clearly  understood  that  the  quality  and  the  extent  of  achievable  super¬ 
resolution  depends  on  the  accuracy  and  the  amount  of  a  priori  information  that  could  be 
utilized  in  the  solution  process.  The  available  information  can  be  used  to  develop  appropriate 
constraint  sets  that  could  steer  the  iterative  estimation  process  in  more  efficient  directions.  In 
particular,  the  goals  for  further  work  should  include  the  following: 

•  development  of  novel  approaches  based  on  convex  constraint  sets  for  resolution 
enhancement  in  the  presence  of  noise  and  clutter,  and  for  analysis  of  the  limits  to  super¬ 
resolution; 

•  mathematical  modeling  of  constraint  sets  and  projection  operators  for  specific  a  priori 
information  about  the  object  or  scene  being  imaged  in  order  to  facilitate  inclusion  in  the 
design  of  systematic  restoration  and  super-resolution  procedures  based  on  the  Projection- 
Onto-Convex-Sets  (POCS)  approach  [43,44]; 

•  investigation  of  methods  for  combining  POCS-based  approaches  with  the  ML  and  MAP 
restoration  methods  discussed  in  this  report  for  designing  hybrid  POCS-Bayesian  super¬ 
resolution  procedures. 
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APPENDIX 


In  this  Appendix,  a  complete  listing  of  the  computer  programs  and  libraries  developed 
in  the  IPDSL  Laboratory  for  conducting  various  super-resolution  experiments  and 
investigations  described  in  this  report  will  be  given.  These  programs  are  written  in  C 
language  to  facilitate  easy  use  of  these  by  others.  A  listing  of  the  names  of  the  various 
programs  and  a  statement  of  what  each  program  is  designed  to  perform  will  be  given  first, 
which  will  be  followed  by  the  detailed  code  for  each  program. 


A.l  List  of  Program  Names,  Libraries  and  Headers 


•  Programs 


Program 

Description 

aperture . cpp 

Creates  an  aperture 

circles . cpp 

Creates  a  test  image  with  circles 

convolve . cpp 

Convolves  two  images 

create . cpp 

Creates  an  empty  image 

cutout . cpp 

Cuts  out  a  smaller  image  from  another 

dimensions . cpp 

Shows  the  dimensions  of  an  image 

extreme . cpp 

Shows  the  extreme  values  of  an  image 

ima2pgm. cpp 

Converts  from  IMA  to  PGM 

insert . cpp 

Inserts  a  small  image  into  another 

magspec . cpp 

Calculates  the  magnitude  of  the  spectrum 

pattern. cpp 

Creates  a  test  image  with  rectangular  patterns 

pgm2ima . cpp 

Converts  from  PGM  to  IMA 

point . cpp 

Creates  a  test  image  with  a  centered  point 

psf . cpp 

Obtains  a  PSF  from  an  aperture 

resample . cpp 

Resamples  an  image  to  any  desired  size 

superresolution . cpp 

Superresolves  an  image 

threshold. cpp 

Thresholds  an  image 

totalf lip . cpp 

Applies  the  flip  operator  to  an  image 

•  Libraries  and  Headers 


Library  or  Header 

Description 

df  t .  h 
dft . cpp 
pgm.h 
pgm.  cpp 
global .h 
image .h 
image . cpp 
mcomplex . h 
mcomplex. cpp 

Header  file  for  dft. cpp 

Routines  needed  to  calculate  the  FFTs 

Header  file  for  pgm. cpp 

Routines  to  convert  to/ from  IMA  to/ from  PGM 
Header  file  with  all  global  constants 

Header  file  for  image . cpp 

Routines  needed  to  manipulate  an  image 

Header  file  for  mcomplex. cpp 

Routines  needed  to  manipulate  complex  numbers 

121 


A.2  Description  of  Code 


NAME 


aperture . cpp 


USAGE 

aperture  szAperturelmaName  inXDim  inYDim  szType  inValuel  inValue2 

INPUT  PARAMETERS  TYPE 


string 

szAperturelmaName 

integer 

inXDim 

integer 

inYDim 

string 

szType 

integer 

inValuel 

integer 

inValue2 

DESCRIPTION 

Creates  an  image  of  an  ellipsoidal  aperture,  <szType>  defines 
has  to  be  equal  to  ELLIPSOIDAL.  The  ellipse's  horizontal  axis 
its  vertical  axis  equal  to  <inValue2>  pixels.  The  aperture . is 
<inXDim>  by  <inYDim>  pixels.  This  image  is  stored  in  the  file 


the  type  of  aperture.  <szType> 
is  equal  to  <inValuel>  pixels , 
stored  in  an  image  with 
<szApertureImaName> . 


PROGRAMMER 

Pablo  Zecrers 


#include  <stdlib.h> 
tinclude  <stdio.h> 

# include  <string.h> 

#include  " . . \ . \Tools\global\global .h" 

# include  " . . \ . \Tools\image\ image .h" 

# include  " . . \ . .  \Tools\mcornplex\mcomplex.h" 

# include  n . . \ . \Tools\df t\df t .h" 

int  main (int  ac,  char  *av[]) 

{ 

char 
int 
IMAGE 

inStatus  =  OK; 
pIMImal  =  NULL; 
if  (ac  !=  7) 

printf ( "USAGE:  aperture  szAperturelmaName  inXDim  inYDim  szType  inValuel  inValue2\n" ) ; 
inStatus  =  NX; 

} 

else 

{ 

if  (strcmp (av[4] , "ELIPSOIDAL" )  ==  0) 

{ 

pIMImal  =  pIMIMAAskImaMem(atoi (av[2 ] ) , atoi (av[3] )  )  ; 

inIMAElipsoidalAperture (pIMImal , atoi (av[5] ) ,  atoi (av[6] ) ) ; 

sprintf (szBuf fer ,  "%s . ima" , av[ 1] ) ; 
inIMASavelma (szBuffer , pIMImal) ; 

} 

} 

if  (pIMImal  ! -  NULL) 

inIMAFreelmaMem (pIMImal ) ; 

return ( inStatus) ; 

} 


szBuf fer [100] ; 
inStatus; 
♦pIMImal; 
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NAME 

circles . cpp 

USAGE 

circles  szMatrixName  inSize 
INPUT  PARAMETERS 

string  szMatrixName 

integer  inSize 

DESCRIPTION 

Creates  a  square  image  of  <inSize>  by  <inSize>  pixels  with  concentric  circles  and  stores  it 
in  the  file  <szMatrixName> . 

PROGRAMMER 

Pablo  Zegers  _ _ 


# include  <stdlib.h> 
iinclude  <stdio.h> 

# include  <string.h> 

# include  <math.h> 

# include  " .  .  \ . . \Tools\global\global .h" 

# include  " . . \ . . \Tools\ image \ image .h" 

# include  " . . \ . . \Tools\mcomplex\mcomplex.h" 
# include  " . . \ . . \Tools\df t\df t .h" 

#def ine  HEIGHT  128 

#def ine  PERIOD  64 


int  main ( int  ac,  char  *av(]) 
{ 

char 

int 


float 

mcomplex 

IMAGE 


szBuf fer [ 100 ] ; 
inStatus, 

inSize, 
inCount 1, 
inCount 2 ; 
flRadius, 

flValue; 
mcBuf fer ; 

* pIMImal; 


inStatus  =  OK; 


pIMImal  =  NULL; 

if  (ac  ! =  3) 

{ 

print f ( "USAGE:  circles  szMatrixName  inSizeNn" ) ; 
inStatus  =  NK; 

} 

else 

{ 

inSize  =  atoi(av[2}); 

pIMImal  =  pIMIMAAsklmaMem (inSize, inSize)  ; 

if  (pIMImal  ==  NULL) 

{ 

printf ( "The  image  could  not  be  created! \n" ) ; 
inStatus  =  NK; 

> 

else 

{ 

for  ( inCountl=0 ; inCount 1< inSize; inCountl++) 

{ 

for  (inCount2=0 ; inCount2<inSize; inCount2++) 

{ 

flRadius  =  (float) sort ( (float) { { inCountl-inSize/2 ) * ( inCount 1- inSize/ 2 ) + (inCount2- 
inSize/2 ) * ( inCount2-inSize/2 ) ) ) ; 

flValue  =  (float) (128*cos( (float) (2 *PI*f lRadius/ ( ( float ) PERIOD) ) ) ) ; 

if  ((flRadius  <  inSize/2)  &&  (flValue  >  127)) 
voSetCom ( fcmcBuf  f er , HEIGHT , 0 ) ; 
else 

voSetCom (&mcBuf fer, 0,0); 

inlMASetlmaElem  (pIMImal,  inCountl,  inCount  2  ,  mcBuf  fer)  ; 

} 
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} 

sprint f (szBuf fer , " %s . ima" , av[13 ) ; 
inIMASavelma (szBuf fer,pIMImal) ; 

inlMAFreelmaMem {pIMImal } ; 

} 

} 

return { inStatus) ? 
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NAME 

convolve . cpp 

USAGE 

convolve  szResult  szlmal  szlma2  szConTyp 
INPUT  PARAMETERS 

string  szResult 

string  szlmal 

string  szlma2 

string  szConTyp 

DESCRIPTION 

Convolves  the  images  stored  in  the  files  <szlmal>  and  <szlma2>.  The  string  <szConTyp>  defines 
the  type  of  convolution:  it  can  be  LINEAR  or  CIRCULAR-  The  resulting  image  is  stored  in  the 
file  <szResult>. 

PROGRAMMER 

Pablo  Zegers  _ _ _ _ _ _ _ 


#include  <stdlib.h> 

#include  <stdio.h> 

# include  <string.h> 

# include  * . . \ . . \Tools\global\global .h" 

# include  * . . \ . . \Tools\ image \ image .h" 

# include  " . . \ . . \Tools\mcomplex\mcomplex.h" 
#include  " . . \ . \Tools\df t\df t .h" 

int  main{int  ac,  char  *av[]) 

{ 

char 
int 
IMAGE 


inStatus  =  OK; 

pIMImal  =  NULL; 
pIMIma2  =  NULL; 
pIMIma3  =  NULL; 

if  (ac  ! =  5) 

printf ( "USAGE:  convolve  szResult  szlmal  szlma2  szConTyp \nM ) ; 
inStatus  =  NK; 

} 

else 

{ 

sprint f (szBuffer, " %s . ima" , av[2] ) ; 
pIMImal  =  pIMIMAReadlma (szBuffer) ; 
if  (pIMImal  ==  NULL) 

{ 

printf ("The  image  %s  could  not  be  read! \nM , av[2 ] ) ; 
inStatus  =  NK; 

} 

sprintf (szBuffer, "%s . ima" ,av[3 ] ) ; 
pIMIma2  =  pIMIMAReadlma ( szBuffer) ; 
if  (pIMIma2  ==  NULL) 

{ 

printf ("The  image  %s  could  not  be  read! \n" , av[3 ] ) ; 
inStatus  =  NK; 

) 

if  (inStatus  !=  NK) 

{ 

pIMIma3  =  pIMIMAAskImaMem(pIMIma2->inNumRows,pIMIma2->inNumCols)  ; 

if  (strcmp ( "LINEAR" , av[4 ) )  ==  0) 
inIMAImaConvolution (pIMIma3 , pIMImal ,pIMIma2, LINEAR) ; 

else  if  (strcmp ( "CIRCULAR" , av[4] )  ==  0) 
inIMAImaConvolution (pIMIma3 , pIMImal ,pIMIma2 , CIRCULAR) ; 
else 

inStatus  =  NK; 

inIMAImaFunOp (pIMIma2 , pIMIma3 , IMRA) ; 
sprintf (szBuffer, "%s . ima" ,av[l] ) ; 


szBuffer [100] ; 
inStatus; 
*pIMImal, 

*pIMIma2 , 
*pIMIma3 ; 
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inIMASavelma ( szBuf f er , pIMIma2 ) 

} 

} 

if  (pIMImal  !=  NULL) 

inIMAFreelmaMem (pIMImal )  ; 
if  (pIMIma2  !=  NULL) 

inIMAFreelmaMem (pIMIma 2 ) ; 
if  (pIMIma3  !=  NULL) 

inIMAFreelmaMem (pIMIma3 ) ? 


return (inStatus) ; 


#include  <stdlib.h> 

#include  <stdio.h> 

#include  <string.h> 

# include  " . . \ . . \Tools\global\global .h" 

# include  *’ .  .  \  .  .  \Tools \image\ image  .h" 

# include  " . . \ . . \Tools\mcomplex\mcomplex .h" 
tinclude  " . . \ . \Tools\df t\df t .h" 

int  main ( int  ac,  char  *av[]) 

{ 

char 
int 
IMAGE 

inStatus  =  OK; 
pi Mima 1  =  NULL; 
if  (ac  ! =  4) 

pr int f ( "USAGE :  create  szImaName  inHorSize  inVerSize\n" ) ; 
inStatus  =  NK; 

} 

else 

pIMImal  =  pIMIMAAskImaMem(atoi (av(2 ] ) , atoi (av[3 ] ) ) ; 

sprintf (szBuf fer, "%s . imafl ,av[l]); 
inIMASavelma (szBuf fer, pIMImal) ; 

} 

if  (pIMImal  ! =  NULL) 

inIMAFreelmaMem (pIMImal) ; 

return (inStatus) ; 

} 


szBuf fer [100] ; 
inStatus; 

* pIMImal ; 


NAME 

cutout .cpp 

USAGE 

cutout  szOutlma  szlnlma  inX  inY  inDeltaX  inDeltaY 

INPUT  PARAMETERS 


string 

szOutlma 

string 

szlnlma 

integer 

inX 

integer 

inY 

integer 

inDeltaX 

integer 

inDeltaY 

DESCRIPTION 

From  the  image  stored  in  the  file  <szlnlma>,  cuts  out  another  image  whose  upper  left  corner 
is  positioned  at  (<inX> , <inY>) .  The  size  of  this  cut  out  image  is  <inDeltaX>  by  <inDeltaY> 
pixels . 

PROGRAMMER 

Pablo  Zegers _ ___ _ _ _ 


#include  <stdlib.h> 

#include  <stdio.h> 

# include  <string.h> 

# include  n . . \ . . \Tools\global\global .h" 

# include  " . . \ . . \Tools\image\ image. h" 

# include  "  .  .  \  .  .  \Tools\mcomplex\mcomplex.h,' 
# include  " . . \ . . \Tools\df t\df t .h" 

#def ine  UP  0 

tdefine  DOWN  1 

int  main{int  ac,  char  *av[}) 

{ 

char 
int 
IMAGE 


inStatus  =  OK; 

pIMImal  =  NULL; 
pIMIma2  =  NULL; 

if  (ac  ! =  7) 

printf ( "USAGE:  cutout  szOutlma  szlnlma  inX  inY  inDeltaX  inDeltaYXn" ) ? 
inStatus  =  NK; 

} 

else 

{ 

sprintf (szBuf fer, "%s . ima" , av[2  3 ) ; 
pIMImal  =  pIMIMAReadlma (szBuf fer) ; 

if  (pIMImal  ==  NULL) 

{ 

printf ("The  image  %s  could  not  be  read! \n" , av[2 ] ) ; 
inStatus  =  NK; 

} 

else 

{ 

pIMIma2  =  pIMIMAAsklmaMem {atoi (av[5 ] ) , atoi (av [6] ) ) ; 

inlMAExtractlma (pXMIma2 , pIMImal , atoi (av[3] ) , atoi(av[4] ) ,atoi (av[5] ) ,atoi (av[6] ) ) ; 

sprintf (szBuf fer, "%s . ima" ,av[l] ) ; 
inIMASavelma (szBuf fer , pIMIma2 ) ; 

} 

} 

if  (pIMImal  !=  NULL) 

inIMAFreelmaMem (pIMImal) ; 
if  (pIMIma2  !=  NULL) 

inIMAFreelmaMem (pIMIma2 ) ; 

return ( inStatus ) ; 

) 


szBuf fer [100] ; 
inStatus ; 
♦pIMImal # 

*pIMIma2 ; 
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NAME 

dimensions . cpp 

USAGE 

dimensions  szlma 

INPUT  PARAMETERS 

string  szlma 

DESCRIPTION 

Prints  out  the  dimensions  of  the  image  stored  in  the  file  <szlma>. 

PROGRAMMER 

Pablo  Zegers  _ 


#include  <stdlib.h> 

#include  <stdio.h> 

# include  " . . \ . \Tools \global\global . h" 

#include  " . . \ . . \Tools\image\image . h" 

# include  " . . \ . . \Tools\mcomplex\mcomplex.h" 

#include  " . . \ . \Tools\df t\df t .h" 

int  main(int  ac,  char  *av[]) 

{ 

char 
int 
IMAGE 

inStatus  =  OK; 

pIMImal  =  NULL; 

if  (ac  ! =  2) 

{ 

printf ( "USAGE :  dimensions  szlma\n" ) ? 
inStatus  =  NK; 

} 

else 

( 

sprintf {szBuffer, "%s.ima" , av[l] ) ; 
pIMImal  =  pIMIMAReadlma (szBuffer) ; 

if  (pIMImal  ==  NULL) 

{ 

printf ("The  image  %s  could  not  be  read! \n" , av[l] ) ; 
inStatus  =  NK; 

} 

else 

{ 

printf  ("  %i  rows  by  %i  col\omns\n" , pIMImal ->inNumRows , pIMImal ~>inNumCols)  ; 

} 

} 

if  (pIMImal  ! =  NULL) 

inIMAFreelmaMem (pIMImal)  ; 

return ( inStatus) ; 

} 


szBuffer [100] ; 
inStatus; 
*pIMImal; 
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NAME 

extreme . cpp 

USAGE 

extreme  szlma  szMinOrMax  szExtTyp 
INPUT  PARAMETERS 

string  szlma 

string  szMinOrMax 

string  szExtTyp 

DESCRIPTION 

Prints  out  the  extreme  value  of  the  image  stored  in  the  file  <szlma>.  The  parameter 
<szMinOrMax>  defines  if  the  extreme  is  a  minimum  or  a  maximum,  and  it  has  to  be  either  MIN  or 
MAX.  The  parameter  <szExtTyp>  has  to  be  MAGNITUDE ,  REAL,  or  IMAGINARY  according  to  the 
extreme  that  needs  to  be  found. 

PROGRAMMER 

Pablo  Zegers 


#include  <stdlib.h> 

#include  <stdio.h> 

# include  <string.h> 

# include  " . . \ . . \Tools\global\global . h" 

# include  " . . \ . . \Tools\image\image .h" 

# include  " . . \ .  \Tools\mcomplex\mcomplex.h" 

#include  " . . \ . \Tools\df t\dft .h" 

int  main { int  ac,  char  *av[]) 

{ 

char 
int 

float 
IMAGE 

inStatus  =  OK; 

pIMImal  =  NULL; 

if  (ac  i =  4) 

{ 

printf ( "USAGE:  extreme  szlma  szMinOrMax  szExtTyp\n" ) ; 
inStatus  =  NK; 

} 

else 

{ 

sprint f ( szBuf fer, "%s . ima" , av[l] ) ; 
pIMImal  =  pIMIMAReadlma (szBuf fer) ; 

if  (pIMImal  ==  NULL) 

{ 

printf ("The  image  %s  could  not  be  read! \n" , av[l]  )  ; 
inStatus  =  NK; 

} 

else 

{ 

if  (strcmp(av[3] , "MAGNITUDE")  ==  0) 
inType  =  MAGNITUDE; 
else  if  {strcmp (av [3] , "REAL" )  ==  0) 
inType  =  REAL; 
else 

inType  =  IMAGINARY; 

if  (strcmp (av[2 ], "MIN" )  ==  0) 

inlmaMin ( &f lExtreme , pIMImal , inType) ; 
else 

inlmaMax (&f lExtreme, pIMImal, inType) ; 
printf ("The  extreme  is  %. 4f\n. ", f lExtreme) ; 

} 

> 

if  (pIMImal  !=  NULL) 

in IMAFreelmaMem (pIMImal)  ; 

return (inStatus)  ; 

} 


szBuf fer [100] ; 
inStatus, 

inType ; 
f lExtreme; 
♦pIMImal; 
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NAME 

ima2pgm.  cpp 

USAGE 

ima2pgm  szDesImaName  szOrilmaName 

INPUT  PARAMETERS 

string  szDesImaName 

string  szOrilmaName 

DESCRIPTION 

Converts  the  PGM  P5  image  stored  in  the  file  <szOriImaName>  to  the  IMA  format.  The  result  is 
stored  in  the  file  <szDesImaName> . 

PROGRAMMER 

Pablo  Zegers  _ 


#include  <stdlib.h> 
tinclude  <stdio.h> 

# include  <string.h> 

# include  " . . \ . . \Tools\global\global .hM 
# include  " . . \ . \Tools \ filters \ filters .h" 

# include  " . . \ . . \Tools\Image\Image . h" 

#include  " . . \ . . \Tools\mcomplex\mcomplex.h" 

# include  " . . \ . . \Tools\df t\df t .h" 

int  main ( int  ac,  char  *av[]) 

{ 

char  szBuf f er [ 100] ; 

int  inStatus; 

IMAGE  *pIMImal, 

*pIMIma2 ; 


inStatus  =  OK; 

pIMImal  =  NULL; 
pIMIma2  =  NULL; 

if  (ac  ! =  3) 

{ 

printf ( "USAGE:  ima2pgm  szDesImaName  szOrilmaNameNn" ) ; 
inStatus  =  NK; 

} 

else 

{ 

sprintf (szBuf fer, " %s . ima" , av[2] ) ; 
pIMImal  =  pIMIMAReadlma (szBuf fer) ; 

if  (pIMImal  ==  NULL) 

{ 

printf("The  image  %s  could  not  be  read! \n" , av[23 ) ; 
inStatus  =  NK; 

} 

else 

{ 

sprintf (szBuf fer, "%s .pgm" , av[l] ) ; 

pIMIma2  =  pIMIMAAsklmaMem (pIMImal->inNumRows , pIMImal->inNumCols )  ; 
inIMAStretch(pIMIma2 , pIMImal  ,0,255)  ; 
inFILSavePGM (szBuf fer ,pIMIma2 , 255)  ; 

} 

} 

if  (pIMImal  !=  NULL) 

inIMAFreelmaMem (pIMImal ) ; 
if  (pIMIma2  !=  NULL) 

inIMAFreelmaMem  (pIMIma2 )  ; 

return (inStatus) ; 
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NAME 

insert . cpp 

USAGE 

insert  szOutlma  szlnlma  inX  inY 
INPUT  PARAMETERS 

string  szOutlma 

string  szlnlma 

integer  inX 

integer  inY 

DESCRIPTION 

Inserts  the  image  stored  in  the  file  <szlnlma>  in  the  image  stored  in  the  file  <sz0utlma>. 

The  upper  left  corner  of  the  image  to  be  inserted  is  positioned  at  (<inX>, <inY>) .  If  some 
portions  of  the  image  to  be  inserted  don't  fit  in  the  target  image,  they  are  clipped  out.  The 
resulting  image  is  stored  in  the  file  <sz0utlma>. 

PROGRAMMER 

Pablo  Zegers _ _ _ 

#include  <stdlib.h> 
tinclude  <stdio.h> 

#include  <string.h> 

# include  11 .  .  \  .  .  \Tools\global\global  .h" 

# include  " . . \ . . \Tools \ image \ image .h" 

# include  " . . \ . \Tools\mcomplex\mcomplex.h" 

# include  " . . \ . . \Tools\df t\df t . hn 

#def ine  UP  0 

#def ine  DOWN  1 

int  main (int  ac,  char  *av[]) 

{ 

char 
int 
IMAGE 


inStatus  =  OK; 

pIMImal  =  NULL; 
pIMIma2  =  NULL; 

if  (ac  !=  5) 

{ 

pr int f ( "USAGE:  insert  szOutlma  szlnlma  inX  inY\n"); 
inStatus  =  NK; 

} 

else 

{ 

sprintf (szBuf fer, "%s . ima" , av[ 1] ) ; 
pIMImal  =  pIMIMAReadlma (szBuf fer) ; 

if  (pIMImal  ==  NULL) 

{ 

printfCThe  image  %s  could  not  be  read! \n" , av[2] ) ; 
inStatus  =  NK; 

> 

else 

{ 

sprintf (szBuf fer, "%s . ima" , av[2] ) ; 
pIMIma2  =  pIMIMAReadlma (szBuf fer) ; 

if  (pIMIma2  ==  NULL) 

{ 

printf("The  image  %s  could  not  be  read! \n" ,av[3]) ; 
inStatus  =  NK; 

} 

else 

( 

inlMAInsertlma (pIMImal , pIMIma2 , atoi (av[3 ] ) , atoi (av[4] ) ) ; 

sprintf (szBuf fer, "%s. ima" , av[l] ) ; 
inIMASavelma (szBuf fer , pIMImal) ; 

} 

} 

} 


szBuf fer [100] ; 
inStatus; 
*pIMImal , 

*pIMIma2 ; 
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if  (pIMImal  !=  NULL) 

inlMAFreelmaMem (pIMImal ) 
if  (pIMIma2  !=  NULL) 

inlMAFreelmaMem (pIMIma2 ) 


return { inStatus) ; 


NAME 

magspec . cpp 

USAGE 

magspec  szMagSpecImaName  szImaName 

INPUT  PARAMETERS 

string  szMagSpecImaName 

string  szImaName 

DESCRIPTION 

Calculates  the  magnitude  of  the  spectrum  domain  of  the  image  stored  in  the  file  <szImaName>. 
The  resulting  image  is  stored  in  the  file  <  szMagSpecImaName> . 

PROGRAMMER 

Pablo  Zegers  _ 


#include  <stdlib.h> 

#include  <stdio.h> 

# include  " . . \ . . \Tools\global\global .h" 

# include  " . . \ . . \Tools\image\ image .h" 

# include  " . . \ . . \Tools\mcomplex\mcomplex.h" 
# include  " . . \ . . \Tools\df t\df t .h" 

int  main{int  ac,  char  *av[]) 

{ 

char 
int 

mcomplex 
IMAGE 


inStatus  =  OK; 

pIMImal  =  NULL; 
pIMIma2  =  NULL; 

if  (ac  !=  3) 

{ 

print f  ( "USAGE :  magspec  szMagSpecImaName  szImaName\n'' )  ; 
inStatus  =  NK; 

} 

else 

{ 

sprint f (szBuf fer, "%s .ima" ,av[2] ) ; 
pIMImal  =  pIMIMAReadlma (szBuf fer) ; 

if  (pIMImal  ==  NULL) 

{ 

printfC’The  image  %s  could  not  be  read!  \n"  ,  av[2  ]  )  ; 
inStatus  =  NK; 

> 

else 

{ 

pIMIma2  =  pIMIMAAsklmaMem (pIMImal ->inNumRows , pIMImal->inNumCols ) ; 
inIMADFTIma (pIMIma2 , pIMImal , DFT) ; 
inIMAImaFunOp (pIMImal , pIMIma2 , MAGNITUDE) ; 
voSetCom (&mcValue, 1,0) ; 

inIMAImaScaOp  (pIMIma2 ,  pIMImal ,  mcValue,  ADD)  ? 
inIMAImaFunOp (pIMImal , pIMIma2 , LOG) ; 
inIMAImaShif t (pIMIma2 , pIMImal ) ; 

sprintf (szBuf fer, "%s . ima" , av[l] ) ; 
inIMASave Ima (szBuf fer, pIMIma2 ) ; 

} 

} 

if  (pIMImal  ! =  NULL) 

inIMAFreelmaMem (pIMImal) ; 
if  (pIMIma2  !=  NULL) 

inIMAFreelmaMem (pIMIma2 ) ; 

return ( inStatus ) ; 

) 


szBuf fer [100] ; 
inStatus ; 
mcValue; 
*pIMImal , 

*pIMIma2 ; 
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NAME 

pattern. cpp 

USAGE 

pattern  szMatrixName  inSize 
INPUT  PARAMETERS 

string  szMatrixName 

integer  inSize 

description 

Creates  a  square  image  of  <inSize>  by  <inSize>  pixels  with  rectangular  patterns  and  stores  it 
in  the  file  <szMatrixName> . 

PROGRAMMER 

Pablo  Zegers  _ _ _ . _ - _ 


# include  <stdlib.h> 
tinclude  <stdio.h> 

# include  <string.h> 

# include  <math.h> 

# include  " . . \ . . \Tools\global\global . h“ 

# include  " . . \ . . \Tools\image\ image .h" 
#include  ■ . . \ . \Tools\mcomplex\mcomplex.h" 
# include  u . . \ . . \Tools\df t\df t .h" 

# define  HEIGHT  128 

#def ine  PIXSHI  28 

int  main(int  ac,  char  *av[]) 

{ 

char 
int 


float 
mcomplex 
IMAGE 

inStatus  =  OK; 
pIMImal  =  NULL; 
if  (ac  ! =  3) 

pr int f ( "USAGE:  pattern  szMatrixName  inSize\nM); 
inStatus  =  NK; 

} 

else 

{ 

inSize  =  atoi(av[2]); 

pIMImal  =  pIMIMAAsklmaMem ( inSize , inSize) ; 

if  (pIMImal  ==  NULL) 

{ 

printf ( "The  image  could  not  be  created! \n" ) ; 
inStatus  =  NK; 

} 

else 

{ 

inWidthl  =  ( int) floor ( inSize/ 5 ) ; 
inWidth2  =  ( int) floor ( inSize/7 ) ; 

for  (inCount 1=0; inCountlcinSize; inCountl++) 

{ 

for  (inCount2=0; inCount2<inSize; inCount2++) 

{ 

if  ( inCount 1  <  inWidthl) 

{ 

f lvalue  =  0; 

else  if  ( ( inCount 1  >=  inWidthl)  &&  (inCountl  <  4 * inWidthl ) ) 

{ 

if  (inCount2  <  inWidth2) 

{ 


szBuf fer [100] ; 
inStatus , 

inSize, 
inCountl, 
inCount2 , 
inWidthl, 
inWidth2 ; 

f lvalue ? 
mcBuf fer; 
♦pIMImal; 
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f lvalue  =  0; 

else  if  { (inCount2  >=  inWidth2)  &&  (inCount2  <  6*inWidth2)) 

{ 

if  ( ( (inCountl  >=  2*inWidthl)  &&  (inCountl  <  3*inWidthl)  &&  {inCount2  > 
2 * inWidth2  +PIXSHI )  &&  (inCount2  <  3*inWidth2+PIXSHI) )  || 

{(inCountl  >=  2*inWidthl)  &&  (inCountl  <  3*inWidthl)  &&  (inCount2  > 
4 * inWidth2 -PIXSHI )  &&  (inCount2  <  5 *inWidth2-PIXSHI ) ) ) 

{ 

f lvalue  =  0; 

} 

else 

{ 

f lvalue  =  HEIGHT; 

} 

} 

else 

{ 

f lvalue  =  0; 

} 

} 

else 

{ 

f lvalue  =  0; 

} 


voSetCom (&mcBuf fer, f lvalue, 0) ; 

inIMASetImaElem(pIMImal, inCountl, inCount2/mcBuffer) ; 

} 

> 

sprint f { szBuf f er , " %s . ima n , av [ 1 ] ) ; 
inIMASavelma (szBuf f er, pIMImal) ; 

inlMAFreelmaMem(pIMImal) ; 

} 

} 

return ( inStatus) ; 

} 
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NAME 

pgm2 ima . cpp 

USAGE 

pgm2ima  szDesImaName  szOrilmaName  | 

INPUT  PARAMETERS 

string 

szDesImaName 

string 

s  z  Or  i  ImaName 

DESCRIPTION 

IMA  image  stored  in  the  file  <szOriImaName>  to  the  PGM  P5  format.  The  result  is 

Converts  the 

stored  in  the 

file  < szDes ImaName > . 

PROGRAMMER 

Pablo  Zegers 

#include  <stdlib.h> 

# include  <stdio.h> 

#include  <string.h> 

# include  . \Tools\global\global .hM 

# include  " . . \ . . \ . . \Tools\f ilters\pgm.h" 

# include  " . . \ . . \ * . \Tools \ image \ image .h" 

#include  " . . \ . . \ . . \Tools\mcomplex\mcomplex.h" 

# include  " . . \ . . \ . . \Tools\df t\df t . h" 

int  main ( int  ac,  char  *av[]) 

{ 

char 
int 
IMAGE 

inStatus  =  OK; 

pIMImal  =  NULL; 

if  (ac  !=  3) 

{ 

pr int f ( "USAGE:  pgm2ima  szDesImaName  szOriImaName\n" ) ; 
inStatus  =  NK; 

} 

else 

{ 

sprintf (szBuf fer, "%s .pgm" , av(2] ) ; 
pIMImal  =  pIMFILReadPGM ( szBuf fer ) ; 

if  (pIMImal  ==  NULL) 

{ 

printfC'The  image  %s  could  not  be  read! \n" , av[2] )  ; 
inStatus  =  NK; 

} 

else 

{ 

sprintf (szBuf fer, "%s.ima" ,av[l) ) ; 
inIMASavelma ( szBuf fer , pIMImal) ; 

> 

} 

if  (pIMImal  !=  NULL) 

inIMAFreelmaMem (pIMImal) ; 

return ( inStatus) ; 

} 


szBuf fer [100] ; 
inStatus; 
♦pIMImal; 
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NAME 

point. cpp 

USAGE 

point  szMatrixName  inSize 
INPUT  PARAMETERS 

string  szMatrixName 

integer  inSize 

DESCRIPTION 

Creates  a  square  image  of  <inSize>  by  <inSize>  pixels  with  a  poit  on  its  center  and  stores  it 
in  the  file  <szMatrixName> . 

PROGRAMMER 

Pablo  Zegers _ _ _ _ _ 


#include  <stdlib.h> 

#include  <stdio.h> 

#include  <string.h> 

# include  <math.h> 

# include  " . . \. . \Tools\global\global . h" 

# include  " . . \ . . \Tools\image\ image .h" 

# include  " . . \ . . \Tools\mcomplex\mcomplex.h" 
# include  " . . \ . . \Tools\df t\df t .h" 

#def ine  HEIGHT  128 

int  main ( int  ac,  char  *av[]) 

{ 

char 
int 


mcomplex 
IMAGE 

inStatus  =  OK; 

pIMImal  =  NULL; 

if  (ac  ! =  3) 

{ 

pr int f ( "USAGE:  point  szMatrixName  inSize\n" ); 
inStatus  =  NK; 

} 

else 

{ 

inSize  =  atoi(av[2)); 

pIMImal  =  p IMIMAAskImaMem( inSize,  inSize)  ; 

if  (pIMImal  ==  NULL) 

{ 

printf ( "The  image  could  not  be  created! \n" ) ; 
inStatus  =  NK; 

} 

else 

{ 

for  (inCount 1=0 ; inCount 1< inSize; inCountl++) 

{ 

for  ( inCount2=0 ; inCount2<inSize; inCount2++) 

{ 

if  ( ( inCount 1  ==  (int) floor (inSize/2) )  &&  (inCount2  ==  { int ) floor { inSize/2 )) ) 
voSetCom ( &mcBuf  f er , HEIGHT ,  0 )  ; 
else 

voSetCom (&mcBuf fer, 0,0) ; 

inlMASetlmaElem (pIMImal , inCountl , inCount2 , mcBuf f er ) ; 

} 

} 

sprintf (szBuf fer , ■ %s . ima" , av[l] ) ; 
inIMASavelma (szBuf fer # pIMImal)  ; 

inIMAFreelmaMem (pIMImal)  ; 

} 

} 


szBuf fer [100] ; 
inStatus, 

inSize, 
inCountl, 
inCount2 ; 
mcBuf fer; 
*pIMImal ; 
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return (inStatus ) ; 
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NAME 

psf .cpp 

USAGE 

psf  szPointSpreadFunction  szAperture  inConType 

INPUT  PARAMETERS 

string  szPointSpreadFunction 

string  szAperture 

integer  inConType 

DESCRIPTION 

Creates  a  point  spread  function  using  the  aperture  defined  by  the  file  <szAperture> .  The  type 
of  convolution  used  is  defined  by  <inConType>,  which  has  to  be  either  LINEAR  or  CIRCULAR.  The 
resulting  image  is  stored  in  the  file  <szPointSpreadFunction> . 

PROGRAMMER 

Pablo  Zegers _ ■  _ ___ _ 


#include  <stdlib.h> 

#include  <stdio.h> 

# include  <string.h> 

#include  \  .  \Tools\global\global  .h" 

# include  " . . \ . . \Tools \image\ image .hn 
tinclude  " . . \ . . \Tools\mcomplex\mcomplex.h" 
#include  " . . \ . . \Tools\dft\dft.h" 

int  main (int  ac,  char  *av[] ) 

{ 

char 
int 

IMAGE 


inStatus  =  OK; 

pIMImal  =  NULL; 
pIMIma2  =  NULL; 

if  (ac  ! =  4) 

{ 

printf ( "USAGE:  psf  szPointSpreadFunction  szAperture  inConType \ n" ) ; 
inStatus  =  NK; 

} 

else 

{ 

sprintf {szBuf fer# " %s . ima" , av[ 2 ] ) ; 
pIMImal  =  pIMIMAReadlma (szBuf fer) ; 

if  { strcmp ( "LINEAR" , av [ 3 ) )  ==  0) 
inConType  =  LINEAR; 

else  if  (strcmp ( "CIRCULAR" , av[3 ] )  ==  0) 
inConType  =  CIRCULAR; 
else 

inStatus  =  NK; 

if  ((inStatus  !=  NK)  &&  (pIMImal  !=  NULL)) 

{ 

pIMIma2  =  pIMIMAAsklmaMem ( pIMImal ->inNumRows , pIMImal ->inNumCols) ; 

inIMAPSFFromAperture (pIMIma2 , pIMImal , inConType) ; 

sprintf (szBuf fer, "%s.ima" ,av[l] ) ; 
inIMASavelma (szBuf fer, pIMIma2 ) ; 

} 

} 

if  (pIMImal  !=  NULL) 

inIMAFreelmaMem (pIMImal) ; 
if  (pIMIma2  !=  NULL) 

inIMAFreelmaMem (pIMIma2 ) ; 

return ( inStatus) ; 

) 


szBuf fer [100] ; 
inStatus , 

inConType ; 

* pIMImal, 

*pIMIma2 ; 
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NAME 

resample.cpp 

USAGE 

resample  szDesImaName  inRows  inCols  szOrilmaName 
INPUT  PARAMETERS 

string  szDesImaName 

integer  inRows 

integer  inCols 

string  szOrilmaname 

DESCRIPTION 

Resamples  the  image  stored  in  the  file  <szOriImaName>  to  another  image  of  size  <inRows>  by 
<inCols>  pixels.  The  resulting  image  is  stored  in  the  file  <szDesImaName> . 

PROGRAMMER 

Pablo  Zegers _ _ _ _ _ 


# include  <stdlib.h> 

#include  <stdio.h> 

#include  " .  .  \  . \Tools\global\global .h" 

# include  " .  .  \  . \Tools \ image \ image .h" 

# include  " .  .  \  . \Tools\mcomplex\mcomplex.h" 

# include  " . . \ . . \Tools\df t\df t . h" 

int  main (int  ac,  char  *av[]) 

{ 

char 
int 
IMAGE 


inStatus  =  OK; 

pIMImal  =  NULL; 
pi Mima 2  =  NULL; 

if  (ac  ! =  5) 

printf ( "USAGE:  resample  szDesImaName  inRows  inCols  szOriImaName\n" ) ; 
inStatus  =  NK; 

} 

else 

{ 

sprint f (szBuf fer, "%s.ima" , av[4] ) ; 
pIMImal  =  pIMIMAReadlma (szBuf fer) ; 

if  (pIMImal  ==  NULL) 

{ 

printf ("The  image  %s  could  not  be  read! \n" , av[4] ) ; 
inStatus  =  NK; 

} 

else 

{ 

pIMIma2  =  pIMIMAAskImaMem(atoi{av[2]),atoi(av[3])); 
inIMAResample (pIMIma2 , pIMImal) ; 

sprint f (szBuf fer , "%s . ima" , av[l] ) ; 
inIMASavelma (szBuf fer, pIMIma2 } ; 

> 

} 

if  (pIMImal  ! =  NULL) 

inIMAFreelmaMem (pIMImal) ; 
if  (pIMIma2  !=  NULL) 

inIMAFreelmaMem (pIMIma2 ) ; 

return (inStatus) ; 

} 


szBuf fer [100] ; 
inStatus; 
*pIMImal , 

*pIMIma2 ; 
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NAME 

superresolution . cpp 

USAGE 

superresolution  szResultName  szIdealName  szBlurred  szPriorName  szPSF  szMethod  szConType 
inUpStep  inUpLev  inRepStep  inNumlter 

INPUT  PARAMETERS 


string 

szResultName 

string 

szIdealName 

string 

szBlurred 

string 

szPriorName 

string 

szPSF 

string 

szMethod 

string 

szConType 

integer 

inUpStep 

integer 

inUpLev 

integer 

inRepStep 

integer 

DESCRIPTION 

inNumlter 

Superresolves  the  image  stored  in  the  file  <szBlurred>.  It  uses  the  images  stored  in  the 
files  <szPriorName>  and  <szPSF>/  as  the  prior  knowledge  image  and  point  spread  function 
needed  for  the  superresolution  process .  The  parameter  <szMethod>  has  to  be  ML  or  MAP 
according  to  the  type  of  superresolution  wanted.  <szConType>  defines  the  type  of  convolution 
and  it  has  to  be  either  LINEAR  or  CIRCULAR.  <inNumIter>  defines  the  number  of  iterations. 
<inUpStep>  defines  the  number  of  iterations  per  each  prograss ive  upsampling  stage.  If 
<inUpStep>  is  equal  to  zero,  then  there  is  not  progressive  upsampling.  <inUpLev>  defines  the 
number  of  pregressive  .upsampling  levels.  <inRepStep>  defines  the  number  of  iterations  that 
have  to  pass  before  the  blurred  image  is  replaced  by  the  superesolved  image.  If  <inRepStep> 
is  equal  to  zero  no  replacement  occurs.  If  <szIdealName>  is  different  from  NULL,  it  should 
define  the  name  of  the  file  that  contains  the  image  to  be  used  as  reference  in  all  the 
reference-dependent  measures.  The  superresolved  image  is  stored  in  the  file  <szResultName> . 

PROGRAMMER 

Pablo  Zegers  _ 


#include  <stdlib.h> 

#include  <stdio.h> 

# include  <string.h> 

# include  " . . \ . . \Tools\global\global .h" 

# include  \ \Tools\image\ image . hn 
#include  "  . .\. . \Tools\mcomplex\mcomplex. h" 

#include  " . . \ . . \Tools\df t\df t .h" 

int  inMeasures { float  *pflLik,  int  inAlgTyp,  IMAGE  *pIMIdeal,  IMAGE  *pIMf,  IMAGE  *pIMg, 

IMAGE  *pIMh,  IMAGE  *pIMp,  int  inConType,  int  inFlag) ; 


int  main (int  ac,  char  *av[]) 
{ 

char 

int 


float 

IMAGE 


inStatus  =  OK; 

pIMIdeal  =  NULL; 
pIMBaseg  =  NULL; 
pIMg  =  NULL; 
pIMp  =  NULL; 
pIMBaseh  =  NULL; 
pIMh  =  NULL; 
pIMf  =  NULL; 


szBuf fer [100 ] ; 
inStatus , 

inConType , 
inCount 1, 
inUpStep, 
inUpLev, 
inRepStep; 

f ILik, 

f lMaxLik; 

♦pIMIdeal , 

♦pIMBaseg, 

♦pIMg, 

♦pIMp, 

♦pIMBaseh, 

♦pIMh, 

♦pIMf , 

♦pIMImal , 
*pIMIma2 , 
♦pIMBest; 
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pIMImal  =  NULL; 
pIMIma2  =  NULL; 
pIMBest  =  NULL; 

if  {ac  ! =  12) 

print f ( "USAGE :  superresolution  szResultName  szIdealName  szBlurred  szPriorName  szPSF  szMethod 
szConType  inUpStep  inUpLev  inRepStep  inNumlterNn" ) ; 
inStatus  =  NK; 

) 

else 

{ 

if  (inStatus  ==  OK) 

{ 

if  (strcmp ( "NULL" , av[2 ] )  !=  0) 

{ 

sprintf ( szBuf fer , " %s . ima" , av[2 ] ) ; 
pIMIdeal  =  pIMIMAReadlma (szBuf fer ) ; 
if  (pIMIdeal  ==  NULL) 

{ 

printfC'The  image  %s  could  not  be  read!  \n" , av[2 ] )  ; 
inStatus  =  NK; 

} 

) 

sprintf (szBuf fer, "%s . ima” , av[3] ) ; 
pIMBaseg  =  pIMIMAReadlma (szBuf fer ) ; 
if  (pIMBaseg  ==  NULL) 

{ 

printfC'The  image  %s  could  not  be  read! \n" , av[3 ] ) ; 
inStatus  -  NK; 

} 

pIMg  =  pIMIMAAskImaMem{pIMBaseg~>inNuinRows , pIMBaseg~>inNumCols ) ; 
inIMACopy Ima (pIMg, pIMBaseg) ; 

sprintf (szBuf fer, "%s . ima" ,av[4] ) ; 
pIMp  =  pIMIMAReadlma (szBuf fer) ; 
if  (pIMp  ==  NULL) 

{ 

printfC'The  prior  %s  could  not  be  read!  \n" ,  av  [4  ]  )  ; 
inStatus  =  NK; 

} 

sprintf (szBuf fer, "%s . imaM , av[5] ) ; 
pIMBaseh  =  pIMIMAReadlma ( szBuf fer)  ; 
if  (pIMBaseh  ==  NULL) 

{ 

printfC'The  prior  %s  could  not  be  read!  \n” ,  av(5]  )  ; 
inStatus  =  NK; 

} 

pIMh  =  pIMIMAAskImaMem(pIMBaseh->inNumRows,pIMBaseh->inNumCols); 
inIMACopylma  (pIMh,  pIMBaseh)  ; 

if  (strcmp ( "ML" ,av[6] )  ==  0) 

{ 

if  ( strcmp ( "LINEAR" , av[7] )  ==  0) 
inConType  =  LINEAR; 

else  if  ( strcmp ( "CIRCULAR", av [7 ] )  ==  0) 
inConType  =  CIRCULAR; 
else 

inStatus  =  NK; 

inMeasures (&flLik,ML, pIMIdeal, pIMg, pIMg, pIMh,pIMp, inConType, NK) ; 

} 

else  if  (strcmp { "MAP" , av[6] )  ==  0) 

{ 

if  (strcmp ( "LINEAR”, av[7] )  ==  0) 
inConType  =  LINEAR; 

else  if  ( strcmp { "CIRCULAR", av [7])  ==  0) 
inConType  =  CIRCULAR; 
else 

inStatus  =  NK; 

inMeasures (&flLik, MAP, pIMIdeal, pIMg, pIMg, pIMh, pIMp, inConType, NK) ; 

} 

else 
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inStatus  =  NK; 


pIMf  =  pIMIMAAsklmaMem  (pIMg->inNumRows , pIMg->inNumCols )  ; 
pIMBest  =  pIMIMAAsklmaMem (pIMg->inNumRows ,pIMg->inNumCols) ; 

inUpStep  =  atoi(av[83); 
inUpLev  =  atoi (av[9) ) ; 
inRepStep  =  atoi (av[ 10) ) ? 

for  (inCountl=0 ; inCountl<atoi {av[ll] ) ; inCountl++) 

if  ( (atoi (av[8) )  !=  0)  &&  (inUpStep  ==  0)  &&  (inUpLev  >  0)) 

{ 

pIMXmal  =  pIMIMAAsklmaMem  ( 2  *pIMg->inNumRows ,  2  *pIMg->inNumCols )  ; 
pIMIma2  =  pIMIMAAskImaMem(2*pIMg->inNumRows,2*pIMg->inNumCols); 

inIMAResample  {pIMImal ,  pIMBaseg)  ; 

inIMAImaFunOp (pIMIma2 , pIMImal , IMRA) ; 
inIMAFreelmaMem (pIMg) ; 

pIMg  =  pIMIMAAsklmaMem  (pIMImal ->inNumRows  ,  pIMImal“>inNumCols)  ; 
inIMACopylma (pIMg,pIMIma2 ) ; 

inIMAResample (pIMImal,  pIMp) ; 

inIMAImaFunOp (pIMIma2 , pIMImal , IMRA) ; 
inIMAFreelmaMem (pIMp) ; 

pIMp  =  pIMIMAAsklmaMem (pIMImal ->inNumRows , pIMImal ->inNumCols)  ; 
inIMACopylma (pIMp,pIMIma2) ; 

inIMAResample (pIMImal , pIMBaseh) ; 

inIMAImaFunOp (pIMIma2 , pIMImal , IMRA) ; 
inIMAFreelmaMem (pIMh) ? 

pIMh  =  pIMIMAAsklmaMem  (pIMImal - > inNumRows ,  pIMImal -> inNumCol s )  ; 
inIMACopylma (pIMh, pIMIma2 ) ; 

inIMAResample (pIMImal , pIMf) ; 

inIMAImaFunOp (pIMIma2 , pIMImal , IMRA) ; 
inIMAFreelmaMem (pIMf) ; 

pIMf  =  pIMIMAAsklmaMem  (pIMImal ->  inNumRows,  pIMImal ->  inNumCols )  ; 
inIMACopylma  (pIMf ,  pIMIma2  )  ; 

inlMT^Resample  (pIMImal  ,pIMBest)  ; 

inIMAImaFunOp (pIMIma2 , pIMImal , IMRA) ; 
inIMAFreelmaMem (pIMBest) ; 

pIMBest  =  pIMIMAAsklmaMem (pIMImal->inNumRows  ,pIMImal->inNumCols)  ? 
inIMACopylma (pIMBest ,pIMIma2 ) ; 

inIMAFreelmaMem (pIMImal) ; 
inIMAFreelmaMem (pIMIma2 ) ; 

inUpStep  =  atoi(av[8]); 
inUpLev  - =  1; 

} 

if  ( (atoi (av[10] )  !=  0)  &&  (inRepStep  ==  0)) 

{ 

inIMACopylma (pIMg, pIMf) ; 

inRepStep  =  atoi (av[ 10] ) ; 

} 

if  ( s trcmp ( " ML ■ , av [ 6 ] )  ==  0) 

{ 

if  ( s  trcmp ( " LINEAR n , av ( 7 ] )  ==  0) 
inConType  -  LINEAR; 

else  if  (strcmp ( "CIRCULAR" ,av[7] )  ==  0) 
inConType  =  CIRCULAR; 
else 

inStatus  =  NK; 

inIMAMaxLik  (pIMf  ,  pIMg,  pIMh,  pIMp,  1 ,  inConType)  ; 
inMeasures  UflLik, ML, pIMIdeal, pIMf , pIMg, pIMh, pIMp,  inConType, OK)  ; 

} 

else  if  (strcmp ( "MAP" , av[ 6] )  ==  0) 

{ 

if  ( strcmp <M LINEAR" ,av [7])  ==  0) 
inConType  =  LINEAR; 

else  if  (strcmp ("CIRCULAR" ,av[7] )  ==  0) 
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inConType  =  CIRCULAR; 
else 

inStatus  =  NK; 

inIMAMAP  (pIMf ,  pIMg, pIMh,  pIMp ,  1 ,  inConType)  ; 

inMeasures ( &f lLik, MAP , pIMIdeal , pIMf , pIMg , pIMh , pIMp , inConType ,  OK)  ; 

} 

else 

inStatus  =  NK; 

if  { inCountl  ==  0) 

{ 

flMaxLik  =  flLik; 
inIMACopylma  (pIMBest ,  pIMf ) ; 

} 

else 

{ 

if  (flMaxLik  <  flLik) 

{ 

flMaxLik  =  flLik; 
inIMACopylma (pIMBest, pIMf ) ; 

} 

} 

inIMACopylma (pIMp, pIMf) ; 

inUpStep-- ; 
inRepStep-- ; 

} 

sprint f (szBuf fer , "best-%s . ima" , av[ 1] ) ; 
inIMASavelma (szBuffer, pIMBest) ; 
sprintf (szBuffer, "%s . ima" , av[l) ) ; 
inIMASavelma (szBuf fer, pIMf ) ; 

} 

} 

if  (pIMIdeal  !=  NULL) 

inIMAFreelmaMem (pIMIdeal)  ; 
if  (pIMBaseg  !=  NULL) 

inIMAFreelmaMem (pIMBaseg) ? 
if  (pIMg  ! =  NULL) 

inIMAFreelmaMem (pIMg) ; 
if  (pIMp  !=  NULL) 

inIMAFreelmaMem (pIMp) ; 
if  (pIMBaseh  !=  NULL) 

inIMAFreelmaMem (pIMBaseh) ; 
if  (pIMh  ! =  NULL) 

inIMAFreelmaMem (pIMh) ; 
if  (pIMf  !=  NULL) 

inIMAFreelmaMem (pIMf) ; 
if  (pIMBest  !=  NULL) 

inIMAFreelmaMem (pIMBest) ; 

return ( inStatus) ; 


int  inMeasures (float  *pflLik,  int  inAlgTyp,  IMAGE  *pIMIdeal,  IMAGE  *pIMf,  IMAGE  *pIMg, 

IMAGE  *pIMh,  IMAGE  *pIMp,  int  inConType,  int  inFlag) 

{ 

int  inStatus, 

inRowSize, 

inColSize; 

float  flLikl, 

f lLik2 , 
f lRes , 
flLPN, 
f ICor, 
f lLoIdEn, 
f lHildEn, 
flLoReEn, 
f lHiReEn, 
flToSi, 
flLoSi, 
flHiSi, 
flVA; 


IMAGE 


pIMImal, 


*pIMIma2 , 
*pIMIma3 , 
*pIMIma4, 
*pIMIma5 ; 


inStatus  =  OK; 

if  (pIMIdeal  !=  NULL) 

{ 

inRowSize  =  pIMIdeal ->inNumRows ; 

inColSize  =  pIMIdeal ->inNumCo Is ; 

} 

else 

{ 

inRowSize  =  pIMg->inNumRows ; 

inColSize  =  pIMg->inNumCols ; 

} 

pIMImal  =  pIMIMAAskImaMem{ inRowSize, inColSize) ; 
inIMAResample (pIMImal ,pIMf ) ; 

pIMIma2  =  pIMIMAAsklmaMem ( inRowSize, inColSize) ; 
inIMAResample (pIMIma2 , pIMg) ; 

pIMIma3  =  pIMIMAAsklmaMem  <pIMh->inNumRows ,  pIMh->inNumRows )  ; 
inIMAImaConvolution  (pIMIma3  ,  pIMh,pIMf ,  inConType)  ; 
pIMIma4  =  pIMIMAAsklmaMem  (pIMh->inNumRows ,  pIMh->inNumRows )  ? 
inIMAImaFunOp  {pIMIma4 ,  pIMIma3  ,  IMRA)  ; 
inIMAFreeImaMem(pIMIma3 ) ; 

pIMIma3  =  pIMIMAAsklmaMem ( inRowSize , inColSize) ; 
inIMAResample (pIMIma3 , pIMIma4 ) ; 
inIMAFreelmaMem (pIMIma4 ) ; 

pIMIma4  =  pIMIMAAsklmaMem (inRowSize, inColSize) ; 
inIMAResample  (pIMIma4 ,  pIMh)  ; 

pIMIma5  =  pIMIMAAsklmaMem (inRowSize, inColSize) ; 
inIMAResample (pIMIma5 , pIMp) ; 

if  (inFlag  ==  NK) 

{ 

flLikl  =  0; 
f lLik2  =  0; 
flRes  =  0; 

} 

else 

{ 

inIMALikelihood(&f lLikl , pIMIma2 ,pIMIma3 , pIMImal, pIMImaS, ML) ; 
inIMALikelihood ( &f lLik2 , pIMIma2 , pIMIma3 , pIMImal , pIMImaS , MAP ) 
inIMAImaMetric ( &f lRes , pIMIma2 , pIMIma3 , LPNORM, 2 ) ; 

} 

if  (pIMIdeal  ==  NULL) 

{ 

flLPN  =  0; 

flCor  =  0; 
f lLoIdEn  =  0; 
f lHildEn  =  0; 
flLoReEn  =  0; 
f lHiReEn  =  0; 
flToSi  =  0; 
flLoSi  =  0; 
flHiSi  =  0; 
flVA  =  0; 

} 

else 

{ 

inIMAImaMetric (&flLPN, pIMIdeal, pIMImal, LPNORM, 2) ; 
inIMAImaMetric (&f ICor , pIMIdeal, pIMImal, CORRELATION, 0) ; 

} 

inIMAFreelmaMem (pIMImal) ; 
inIMAFreelmaMem  (pIMIma2 )  ,* 
inIMAFreelmaMem (pIMIma3 ) ; 
inIMAFreelmaMem (pIMIma4) ; 
inIMAFreelmaMem (pIMImaS) ; 
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print f { "%+e, %+e, %+e, %+e, %+e, %+e, %+e, %+e, %+e, %+e, %+e\n" , 
f  iLikl ,  f  lLik.2  ,  f  IRes ,  f 1LPN,  flCor)  ; 

if  { inAlgTyp  ==  ML) 

( *pf ILik)  =  f ILikl ; 
else  if  (inAlgTyp  ==  MAP) 

{ *pf ILik)  =  f lLik2 ; 

return ( inStatus ) ; 
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NAME 

threshold. cpp 

USAGE 

threshold  szOutlma  szlnlma  szDirection  flThreshold 
INPUT  PARAMETERS 

string  szOutlma 

string  szlnlma 

string  szDirection 

float  flThreshold 

DESCRIPTION  ^ 

Thresholds  the  image  stored  in  the  file  <szlnlma>.  If  <szDirection>  is  UP,  all  pixels  with  a 
value  greater  than  <f lThreshold>  are  assigned  a  value  equal  to  <f lThreshold> .  If 
<szDirection>  is  DOWN  is  the  reverse.  The  resulting  image  is  stored  in  the  file  <sz0utlma>. 

PROGRAMMER 

Pablo  Zegers  _ _ __ _ 


# include  <stdlib.h> 

#include  <stdio.h> 

# include  <string.h> 

# include  “ . . \ . . \Tools\global\global ,h" 
tinclude  " . . \ . . \Too Is \image\ image. h" 

# include  " .  .  \  .  .  \  Tools \mcomplex\mcomplex.h,, 
# include  " . . \ . . \Tools\df t\df t .h” 


#define  UP 
#define  DOWN 

int  main { int  ac,  char  *av[]} 

{ 

char 

int 


float 

mcomplex 

IMAGE 


0 

1 


szBuf fer [100 ] ; 
inStatus, 

inDirection, 
inCount 1 , 
inCount 2 ; 
flThreshold, 

f 1 Imaginary; 

mcValue ; 

♦pIMImal, 

*pIMIma2 ; 


inStatus  =  OK; 

pIMImal  =  NULL; 
pIMIma2  =  NULL; 

if  (ac  ! =  5) 

{ 

pr int f ( "USAGE:  threshold  szOutlma  szlnlma  szDirection  f lThreshold\n" ) ; 
inStatus  =  NK; 

} 

else 

{ 

sprintf (szBuf fer, "%s . ima" ,av[2] ) ; 
pIMImal  =  pIMIMAReadlma (szBuf fer) ; 

if  (pIMImal  ==  NULL) 

{ 

print f ( "The  image  %s  could  not  be  read! \nM , av[2] ) ; 
inStatus  =  NK; 

> 

else 

{ 

if  (strcmp (av[3 ] , "UP" )  ==  0} 
inDirection  =  UP; 

else  if  (strcmp (av [3 }, "DOWN" )  ==  0) 
inDirection  =  DOWN; 
else 

inDirection  =  MERROR; 

if  (inDirection  ==  MERROR) 

{ 

inStatus  =  NK; 

} 

else 

{ 
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f IThreshold  =  ( float) atof (av[4 ])  ; 


pIMIma2  =  pIMIMAAsklmaMem ( pIMImal ->inNumRows , pIMImal- >inNumCols) ; 

for  { inCountl=0 ; inCount l<pIMImal->inNumRows ; inCountl++ ) 

{ 

for  ( inCount2=0 ;  inCount2<pIMImal->inNumCols ;  inCount2++) 

{ 

inlMAGetlmaElem  ( fcmcValue ,  pIMImal ,  inCountl ,  inCount2 )  ; 

if  (inDirection  ==  UP) 

{ 

if  (mcValue . f lRe  >  f IThreshold) 

{ 

fllmaginary  =  mcValue . f llm; 

voSetCom(&mc Value, f IThreshold, fllmaginary) ; 

} 

inlMASetlmaElem {pIMIma2 , inCountl , inCount2 , mcValue ) 

} 

else 

{ 

if  (mcValue . flRe  <  f IThreshold) 

{ 

fllmaginary  =  mcValue.fllm; 

voSetCom(&mcValue, f IThreshold, fllmaginary) ; 

} 

inlMASetlmaElem (pIMIma2 , inCountl, inCount2 , mcValue) 

} 

} 

} 

sprintf (szBuf fer, "%s . iman ,av[l]); 
inIMASavelma (szBuf fer,pIMIma2) ; 

} 

} 


if  .{pIMImal  i  =  NULL) 

inIMAFreelmaMem (pIMImal) ; 
if  (pIMIma2  !=  NULL) 

inIMAFreelmaMem (pIMIma2) ; 


return ( inStatus) ; 


NAME 

totalflip.cpp 

USAGE 

totalflip  szOutlma  szlnlma 

INPUT  PARAMETERS 

string  szOutlma 

string  szlnlma 

DESCRIPTION 

Flips  the  image  stored  in  the  file  <szlnlma>  one  time  horizontally,  and  one  time  vertically. 
The  resulting  image  is  stored  in  the  file  <sz0utlma>. 

PROGRAMMER 

Pablo  Zegers  _  _ 


#include  <stdlib.h> 

#include  <stdio.h> 

#include  " . . \ . . \Tools\global\global .hn 
# include  " . . \ . \Tools\image\ image .h" 

#include  " . . \ . \Tools\mcomplex\mcomplex.h" 

# include  " . . \ . . \Tools\df t\dft.h" 

int  main ( int  ac,  char  *av[]) 

{ 

char  szBuf fer [100] ; 

int  inStatus; 

IMAGE  *pIMImal, 

*pIMIma2 , 

*pIMIma3 ; 

inStatus  =  OK; 

pIMImal  =  NULL; 
pIMIma2  =  NULL; 
pi Mima 3  =  NULL; 

if  (ac  ! =  3) 

{ 

pr int f ( "USAGE:  totalflip  szOutlma  szlnlma\n" )  ; 
inStatus  =  NK; 

} 

else 

{ 

sprint f (szBuf fer, "%s . ima" , av[2 ] ) ; 
pIMImal  =  pIMIMAReadlma (szBuf fer) ; 

if  (pIMImal  ==  NULL) 

{ 

printfC'The  image  %s  could  not  be  read!  \n" ,  av[2]  )  ; 
inStatus  =  NK; 

} 

else 

{ 

pIMIma2  =  pIMIMAAskImaMem(2*pIMImal->inNumRows,  2*pIMImal->inNumCols)  ; 
inIMATotalFlip (pIMIma2 , pIMImal) ; 

pIMIma3  =  pIMIMAAskImaMem(pIMIma2->inNumRows,pIMIma2->inNumCols)  ; 
inIMAImaFunOp (pIMIma3 , pIMIma2 , IMRA) ; 

sprintf ( s  zBuf  f er , " %s . ima  n,av[l]); 
inIMASavelma (szBuf fer , pIMIma3 ) ; 

} 

} 

if  (pIMImal  !=  NULL) 

inIMAFreelmaMem (pIMImal) ; 
if  (pIMIma2  !=  NULL) 

inIMAFreelmaMem (pIMIma2 ) ; 
if  (pIMIma3  !=  NULL) 

inIMAFreelmaMem (pIMIma3) ; 

return ( inStatus) ; 

} 
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#ifndef  FNDFT 
// - 


// 


//  INCLUDE  FILES 
// - 


# include  " . . \Global\Global .h" 

# include  " . . \mcomplex\mcomplex.h" 

// - 


//  CONSTANTS 
// - 


#def ine  DFT 
#define  I DFT 

II - 

//  PUBLIC  PROTOTYPES 

II - 

void  vo DFTNDFT ( float  *pflData,  unsigned  long  *pulDims/  int  inNumDim,  int  inOpTy) ; 

- - 


-1 


+  1 


# define  FNDFT  0 

#endif 

// - 
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NAME 

dft.cpp 

DESCRIPTION 

Routines  needed  to  calculate  the  FFTs . 

PROGRAMMER 

Pablo  Zegers  _ 


// 


//  DESCRIPTION  :  Functions  to  perform  FTs  in  n-dimensions  arrays 
//  AUTHOR  :  Pablo  Zegers 

//  DATE  :  February,  1999 

// - 


// 


//  INCLUDE  FILES 
// - 


#include  <stdlib.h> 
#include  <stdio.h> 
#include  <string.h> 
# include  <malloc.h> 
# include  <math.h> 

# include  "dft.h" 


// 


//  CONSTANTS 

//  - 

#def ine  SWAP (a, b) 

tempr= (a) ; (a) = (b) ; (b) =tempr 

//  -  - 

//  INCLUDE  FILES 
// - 


int  inIsPower0f2 (int  inNumber) ; 

void  voNDFFT  { float  *pflData,  unsigned  long  *pulDims,  int  inNumDim,  int  inOpTy); 
void  voNDFT {float  *pflData,  unsigned  long  * pul Dims ,  int  inNumDim,  int  inOpTy); 
inlnitializelndex ( int  *pinVecInd,  int  inNumlnd); 

inlncrementlndex ( int  *pinVecInd,  unsigned  long  *pulDims,  int  inNumlnd); 

II - 


//  PUBLIC  ROUTINES 
// - 


void  voDFTNDFT ( float  *pflData,  unsigned  long  *pulDims,  int  inNumDim,  int  inOpTy) 
{ 

int  inFlag, 

inCountl, 
ulTotalValues ; 


inFlag  =  YES; 

for  ( inCount 1=0; inCount 1< inNumDim; inCountl++) 

{ 

if  (inlsPowerOf 2 (pulDims [inCountl] )  !=  YES) 

inFlag  =  NO; 

} 

if  (inFlag  ==  YES) 

voNDFFT  (pf  lData,pulDims,  inNumDim,  inOpTy)  ; 
else 

voNDFT  (pflData,  pulDims ,  inNumDim,  inOpTy)  ; 

if  (inOpTy  ==  IDFT) 

{ 

ulTotalValues  =  1; 

for  (inCountl=0; inCountl< inNumDim; inCountl++) 
ulTotalValues  *=  pulDims [ inCountl] ; 
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for  ( inCount 1=0 ; inCountl<2*ulTotalValues ; inCountl++) 
pf lData [inCountl]  /=  ulTotalValues ; 

} 

} 

ft - 


//  PRIVATE  ROUTINES 

II - 


// 

// 


int  inIsPowerOf2 (int  inNumber) 

{ 

float  f lResult ; 

f lResult  =  (float) inNumber; 

while  ( f lResult  >  1) 
f lResult  /-  2; 

if  (flResult  ! =  1) 
return (NO) ; 
else 

return (YES) ; 


// 


- - 

void  voNDFFT ( float  *pf lData,  unsigned  long  *pulDims,  int  inNumDim,  int  inOpTy) 
{ 

int  ndim, 

isign, 

idim; 

unsigned  long  il, 

i2  / 
i3 , 

i2rev# 
i3rev, 
ipl  > 

ip2 , 
ip3, 
ifpl/ 
ifp2 , 
ibit, 
k.1 , 
k.2 , 
n, 

nprev, 

nrem, 

ntot; 

unsigned  long  *nn; 

float  *data; 

float  tempi , 

tempr , 
theta, 
wi, 
wpi  # 
wpr, 
wr , 

wtemp; 

//  modification  to  the  original  algorithm 

data  =  pf lData- 1; 

nn  =  pulDims-1; 

ndim  =  inNumDim; 

isign  =  inOpTy; 


for  <ntot=l , idim=l;idim<=ndim;idim++) 
ntot  *=  nn[idim]; 
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nprev  =  1 ; 

for  (idim=ndim; idim>=l; idim--) 

{ 

n  =  nn[idim]; 

nrem  =  ntot/ (n*nprev) ; 
ipl  =  nprev<<l; 
ip2  =  ipl*n; 
ip3  =  ip2*nrem; 
i2rev  =  1; 

for  (i2=l; i2<=ip2 ; i2+=ipl) 

{ 

if  { i2  <  i2rev) 

{ 

for  { il=i2 ; il<=i2+ipl-2 ; il+=2 ) 

{ 

for  (i3=il ; i3<=ip3 ; i3+=ip2 ) 

{ 

i3rev  =  i2rev+i3-i2; 

SWAP (data [i3 ] , data [i3 rev] ) ; 

SWAP (data [i3+l] , data [i3rev+l] ) ; 

} 

} 

} 

ibit  =  ip2  »  1; 

while  ((ibit  >=  ipl)  &&  (i2rev  >  ibit)) 

{ 

i2rev  -=  ibit; 
ibit  >>=  1; 

} 

i2rev  +=  ibit; 

} 

if pi  =  ipl? 
while  (ifpl  <  ip2) 

{ 

ifp2  =  ifpl  <<  1; 

theta  =  ( (float) <isign*6. 28318530717959/ (ifp2/ipl) )) ;  . 

wtemp  =  ( float ) sin (0 . 5* theta) ; 

wpr  =  ( ( f loat) (-2 . 0* wtemp* wtemp) ) ; 

wpi  =  ( float) sin ( theta) ; 

wr  =  1.0; 

wi  =  0.0; 

for  ( i 3  =  1 ; i3<=ifpl; i3+=ipl) 

{ 

for  (il=i3 ; il<=i3+ipl-2; il+=2) 

{ 

for  (i2=il?i2<=ip3?i2+=ifp2) 

{ 

kl  =  i2  ; 

k2  =  kl+ifpl; 

tempr  =  (f loat) wr*data [k2 ]- (float) wi*data[k2+l] ; 

tempi  =  (float)wr*data[k2+l]+(float)wi*data[k2] ; 

data[k23  =  data [kl] -tempr ; 

data[k2+l]  =  data[kl+l]-tempi; 

datalkl]  +=  tempr; 

data[kl+l]  +=  tempi? 

} 

} 

wr  =  (wtemp  =  wr) *wpr-wi*wpi+wr ; 
wi  =  wi*wpr+wtemp*wpi+wi? 

} 

ifpl  =  ifp2? 

} 

nprev  *=  n? 

} 

} 
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// 


void  voNDFT( float  *pflData,  unsigned  long  *pulDims,  int  inNumDim,  int  inOpTy) 

{ 

int  *pinDaInd, 

*pinReInd; 

int  inCount 1, 

inCount2 ; 

unsigned  long  ulTotalValues, 

ulPointPosl , 
ulPointPos2 ; 

float  *pflResult; 

float  flBuffer; 

mcomplex  mcBuf ferl, 

mcBuf fer2 , 
mcBuf f er3 , 
mcBuf fer4; 

//  calculates  the  number  of  real  values  needed  to  store  the  resulting  matrix 
ulTotalValues  =  1; 

for  (inCountl=0; inCountl<inNumDim; inCountl++) 
ulTotalValues  *=  pulDims [ inCountl] ; 

//  asks  memory  to  store  the  vector  of  data  indexes 
pinDalnd  =  (int  *)  calloc (inNumDim, sizeof ( int) ) ; 

//  asks  memory  to  store  the  vector  of  result  indexes 
pinRelnd  =  (int  *)  calloc (inNumDim, sizeof ( int) ) ; 

//  asks  memory  to  store  the  results 

pf iResult  =  (float  *)  calloc ( 2 *ulTotalValues , sizeof ( float )) ; 

inlnitializelndex (pinRelnd, inNumDim) ; 
ulPointPosl  =  0; 
do 
{ 

voSetCom { &mcBuf f erl ,0,0)  ; 
inlnitializelndex (pinDalnd, inNumDim) ; 
ulPointPos2  =  0; 
do 

{ 

flBuffer  =  0; 

for  (inCount 2 =0;inCount2< inNumDim; inCount2++) 
flBuffer  += 

( ( float) pinRelnd [ inCount2 } ) * ( (float) pinDalnd [inCount2] ) / ( ( float) pulDims [inCount2] ) 
voSetCom (&mcBuffer2 , 0 , (float) ( inOpTy*2 *PI* flBuffer ) ) ; 
voExpCom(&mcBuf fer3 , mcBuf fer2 ) ; 

voSetCom { &mcBuffer2 ,pf lData [2 *ulPointPos2 3 ,pf lData [2*ulPointPos2  +  l] ) ; 
voMulCom(&mcBuf fer4 , mcBuf fer2 , mcBuf fer3 ) ; 
voAssignCom (&mcBuf f er2 , mcBuf ferl)  ; 
voAddCom ( &mcBuf f erl , mcBuf f er2 , mcBuf fer4 ) ; 

inlncrementlndex (pinDalnd, pulDims , inNumDim) ; 
ulPointPos2++; 

}  while  (ulPointPos2  <  ulTotalValues); 

pf IResult [2 *ulPointPosl]  =  mcBuf ferl . flRe ; 
pf IResult [2*ulPointPosl+l]  =  mcBuf ferl . film; 

inlncrementlndex (pinRelnd, pulDims , inNumDim) ? 
ulPointPosl++; 

}  while  (ulPointPosl  <  ulTotalValues) ; 

//  copies  the  result  vector  into  the  data  vector 
for  (ulPointPosl=0;ulPointPosl<2*ulTotalValues;ulPointPosl++) 
pf lData [ulPointPosl]  =  pf IResult [ulPointPosl] ; 

//  frees  the  memory  used  to  store  the  data  indexes 
free (pinDalnd) ; 

//  frees  the  memory  used  to  store  the  result  indexes 
free (pinRelnd) ; 

//  frees  the  memory  used  to  store  the  results 
free (pf IResult) ; 
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// 


// - 

inlnitializelndex (int  *pinVecInd,  int  inNumlnd) 

{ 

int  inCountl ; 

for  ( inCountl =0; inCountl<inNumInd; inCountl++) 
pinVecInd  [  inCountl]  =  Gy¬ 
re  turn  (OK)  ; 

} 

// - 


// - 

inlncrementlndex{int  *pinVecInd,  unsigned  long  *pulDims,  int  inNumlnd) 

{ 

int  inFlag, 

inCountl ; 

inFlag  =  YES; 

for  ( inCount l=inNumInd-l; inCount 1>=0; inCountl — ) 

{ 

if  (inFlag  ==  YES) 

pinVecInd [ inCountl]  +=1; 

if  (pinVecInd [ inCountl]  ==  (( int) pulDims [inCountl] ) ) 

{ 

inFlag  =  YES; 

pinVecInd [inCountl]  =  0; 

} 

else 

{ 

inFlag  =  NO; 

} 

} 

return (OK) ; 

} 
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NAME 

pgm.h 

DESCRIPTION 

Header  file  for  pgm.cpp 
PROGRAMMER 

Pablo  Zegers  _ 


// - 

#ifndef  FPGM 

// - 

l! - 

//  INCLUDE  FILES 

// - 

#include  " . . \Global\Global .h" 

# include  " . . \mcomplex\mcomplex.h" 

#include  " . . \ Image \ Image . h " 

II - 

//  CONSTANTS  DEFINITIONS 

// - 

#define  P2  0 

#def ine  P5  1 

#def ine  MAXVAL  255 

// - 

//  PUBLIC  PROTOTYPES 

// - 

IMAGE  *pIMFILReadPGM ( char  ♦szPGMFile) ; 

int  inFILSavePGM (char  *szPGMFile/  IMAGE  *pIMIma,  int  inMaxVal) ; 

H - 

#def ine  FPGM  0 
#endif 

// - 
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NAME 

pgm.  cpp 

description 

Routines  to  translate  an  image  from  the  PGM  format  to  the  IMA  format,  and  vice  versa 

PROGRAMMER 

Pablo  Zegers _ _ _ _ 

- - 

//  DESCRIPTION  :  Functions  to  manipulate  PGM  images 
//  AUTHOR  :  Pablo  Zegers 

//  DATE  :  October,  1997 

// - 

// - - - 

//  INCLUDE  FILES 

- - 

#include  <stdlib.h> 

#include  <stdio.h> 

#include  <string.h> 

# include  <malloc.h> 

# include  <math.h> 
iinclude  <io.h> 

#include  <fcntl.h> 

#include  <sys/stat.h> 

# include  "pgm.h" 

// - 

//  PUBLIC  ROUTINES 

- - 

- - 

- - 

IMAGE  *pIMFILReadPGM(char  *szPGMFile) 

{ 

unsigned  char  ucBuffer; 

char  szBuf fer [MSTRING] , 

szMagic [MSTRING] ; 

int  inStatus, 

inHandle, 
inNumRows , 
inNumCols, 
inMaxVal , 
inCount 1, 
inCount 2 ; 

IMAGE  *pIMIma; 

inStatus  =  OK; 
pIMIma  =  NULL; 

inHandle  =  open (szPGMFile, 0_RD0NLY | 0_B INARY ) ; 
if  (inHandle  <  0) 

{ 

printf ("FILE  % s  COUDN ' T  BE  OPENED . \n" , szPGMFile) ; 
inStatus  =  NK; 

} 

else 

{ 

read (inHandle,  (void*) szBuf fer , 15*sizeof (char) ) ; 
szBuffer[13]  =  0; 

sscanf (szBuf fer, "%s%d%d%d" , szMagic, fcinNumCols , &inNumRows, &inMaxVal) ; 

if  (  (strcmp  (szMagic,  *,P5B )  !=  0)) 

{ 

printf ( "WRONG  FILE  FORMAT . \n" ) ; 
inStatus  =  NK; 

} 

else 

{ 

pIMIma  =  pIMIMAAsklmaMem (inNumRows,  inNumCols)  ; 

if  (pIMIma  ==  NULL) 

{ 

inStatus  =  NK; 
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} 

else 

{ 

for  { inCountl=0 ; inCount l<inNumRows ; inCountl++) 

{ 

for  (inCount2=0; inCount2<inNumCols ; inCount2++) 

{ 

read (inHandle, (void* ) &ucBuffer, sizeof (char )) ; 

voSetCom ( & (pIMIma->ppmcElem [ inCount 1 ] [inCount2] ) , ( (float) ( (int) (ucBuffer) ) ) ,0) ; 

} 

} 

} 

} 

close (inHandle) ? 

} 

return (pIMIma) ; 

} 

// - 

// - 

int  i nF I L Save PGM (char  *szPGMFile,  IMAGE  *pIMIma,  int  inMaxVal) 

{ 


char 

szBuf fer [MSTRING] 

int 

inStatus, 

inHandle, 

inValues, 

inCountl, 
inCount2 ; 

FILE 

♦pFIStream; 

inStatus  =  OK; 

//  inHandle  =  creat (szPGMFile,_S_IREAD |_S_IWRITE) ; 
pFIStream  =  f open ( szPGMFile , "w" ) ; 
if  (inHandle  ==  -1) 

{ 

printf ( " IT  CAN’T  OPEN  FILE  %s\n" , szPGMFile) ; 
inStatus  =  NK; 

} 

else 

{ 

fprintf (pFIStream, " %s\n%d\n%d\n%d\n" , "P2 ",  pIMIma- >inNumCols,  pIMIma ->inNumRows,  inMaxVal) ; 

/ /  sprintf (szBuf fer, "%s\n%d\n%d\n%d\n" , "P2 " , pIMIma->inNumCols/ pIMIma- >inNumRows, inMaxVal) 
//  write (inHandle, (void  * ) szBuf f er , strlen ( szBuf f er ) ) ; 

inValues  =  0 ; 

for  ( inCount 1=0 ; inCount l<pIMIma->inNumRows ; inCount 1++) 

{ 

for  ( inCount2=0 ; inCount2<pIMIma->inNumCols ; inCount2++) 

{ 

if  (inValues  <  40) 

{ 

inValues ++ ; 

> 

else 

{ 

inValues  =  0? 

fprintf (pFIStream, "\nM ) ; 

} 

fprintf (pFIStream, "%i  ", (unsigned  char) ( floor (pIMIma->ppmcElem[ inCountl] [inCount2] . flRe) ) ) ; 
//  szBuffer[0)  =  (unsigned  char) ( floor (pIMIma->ppmcElem [inCountl ][ inCount2 ]. flRe) ) ; 

//  write (inHandle, (void  *) szBuf fer, 1) ; 

} 

} 

} 

//  close { inHandle) ; 
fclose (pFIStream) ; 

return (inStatus) ; 

} 
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NAME 

global. h 

DESCRIPTION 

Defines  all  the  global  constants. 

PROGRAMMER 

Pablo  Zegers  _ _ 


// - 

//  DEFINITIONS 

// - 

tdef ine  OK 
#define  NK 

#def ine  YES 
#define  NO 

# define  MSTRING 
# define  MERROR 

tdefine  IPORT 
#def ine  OPORT 
#def ine  NEURON 
# define  SYSTEM 

#define  PI 

tdefine  PRECISION 


0 

-1 

0 

-1 

100 

-1 

0 

1 

2 

3 

3.1415926 

6 
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#ifndef  F IMAGE 
// - 


// 


//  INCLUDE  FILES 
// - 


#include  " . . \Global\Global .  h" 

# include  " . . \mcomplex\mcomplex.hn 
#include  " . . \df t\df t .h" 

// - 


//  CONSTANTS  DEFINITIONS 
// - 


# define  ADD  0 

# define  SUB  1 

#def ine  MUL  2 

# define  DIV  3 

#def ine  REAL  0 

#def ine  IMAGINARY  1 

#def ine  ZERE  2 

#def ine  ZEIM  3 

#def ine  IMRA  4 

# define  MAGNITUDE  5 

#def ine  PHASE  6 

# define  EXP  7 

# define  LOG  8 

#def ine  POW  9 

#def ine  LPNORM  0 

#def ine  CORRELATION  1 

tdefine  CORNERED  0 

#def ine  CENTERED  1 

#def ine  SPREAD  2 

#def ine  RECTANGULAR  0 

#def ine  HAMMING  1 

tdefine  HANNING  2 

tdefine  BLACKMAN  3 

tdefine  LINEAR  0 

tdefine  CIRCULAR  1 

tdefine  CLS  0 

tdefine  ML  1 

tdefine  MAP  2 


// 


//  STRUCTURE  DEFINITIONS 
// - 


typedef  struct 

{ 

int 


mcomplex 
}  IMAGE; 


in Index, 

inNumRows , 
inNumCols; 


**ppmcElem; 
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// 


//  PUBLIC  PROTOTYPES 
// - 


IMAGE  *pIMIMAAskImaMem(int  inNumRows,  int  inNumCols) ; 
int  inlMAFreelmaMem ( IMAGE  *pIMIma) ; 
int  inlMASetlmalndex { IMAGE  *pIMIma,  int  inindex); 
int  inlMAGetlmalndex { int  *pinlndex,  IMAGE  *pIMIma) ; 

int  inlMASetlmaElem ( IMAGE  *pIMIma,  int  inRow,  int  inCol,  mcomplex  mcValue); 
int  inlMAGetlmaElem (mcomplex  *pmcValue,  IMAGE  *pIMIma,  int  inRow,  int  inCol); 

IMAGE  *pIMIMAReadIma {char  *szName) ; 

int  inIMASavelma (char  *szName,  IMAGE  *pIMIma) ; 

int  inlmaMaxt float  *pflMax,  IMAGE  *pIMIma,  int  inType); 

int  inImaMin( float  *pflMin,  IMAGE  *pIMIma,  int  inType); 

int  inIMAImaMean (mcomplex  *pmcMean/  IMAGE  *pIMIma) ; 

int  inIMAImaVar (mcomplex  *pmcVar,  IMAGE  *pIMIma) ; 

int  inIMAInsertIma( IMAGE  *pIMIma,  IMAGE  *pIMSub,  int  inRowIn,  int  inColIn) ; 

int  inlMAExtractlma { IMAGE  *pIMSub,  IMAGE  *pIMIma,  int  inRowIn,  int  inColIn,  int  inDelRows,  int 
inDelCols ) ; 

int  inIMACopylma ( IMAGE  *pIMDesIma,  IMAGE  *pIMOriIma) ; 
int  inIMAHorFlipIma { IMAGE  *pIMOutIma/  IMAGE  *pIMInIma) ; 
int  inIMAVerFlipIma (IMAGE  *pIMOutIma/  IMAGE  *pIMInIma) ; 
int  inIMAResample ( IMAGE  *pIMOutIma#  IMAGE  *pIMInIma) ; 

int  inIMAImaScaOp ( IMAGE  *pIMImaB,  IMAGE  *pIMImaA,  mcomplex  mcValue,  int  inOpType) ; 

int  inIMAImaMatOp ( IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  ^pIMImaB,  int  inOpType); 

int  inIMAImaFunOp ( IMAGE  *pIMImaB/  IMAGE  *pIMImaA/  int  inOpType); 

int  inIMADFTIma (IMAGE  *pIMOutIma,  IMAGE  *pIMInIma,  int  inTransf ormType) ; 

int  inIMAIsPowerOf 2 (int  inNumber) ; 

int  inIMAGetNextPower0f2 (int  inNumber); 

int  inIMACLS( IMAGE  *pIMf,  IMAGE  *pIMg,  IMAGE  *pIMh,  float  flGamma) ; 

int  inIMAMaxLik( IMAGE  *pIMf,  IMAGE  *pIMg,  IMAGE  *pIMh,  IMAGE  *pIMPrior,  int  inNumlter, 

int  inType) ; 

int  inIMAMAP (IMAGE  *pIMf,  IMAGE  *pIMg,  IMAGE  *pIMh,  IMAGE  *pIMPrior,  int  inNumlter, 

int  inType) ; 

int  inIMAEl ipsoidalAperture ( IMAGE  *pIMAperture,  int  inAxisl#  int  inAxis2) ; 

int  inIMASlantedStripAperture (IMAGE  *pIMAperture ,  int  inLength,  int  inWidth,  float  flAngle) ; 
int  inIMAImaConvolution ( IMAGE  *pIMImaC/  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inConType) ; 

int  inIMAImaCorrelation ( IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inCorType) ; 

int  inIMAOTFFromAperture ( IMAGE  *pIMOTF,  IMAGE  *pIMAperture,  int  inConType); 

int  inlMAPSFFromAperture ( IMAGE  *pIMPSF,  IMAGE  *pIMAperture ,  int  inConType); 

int  inIMAStretch( IMAGE  *pIMOutIma,  IMAGE  *pIMInIma,  int  inMinVal,  int  inMaxVal) ; 
int  inIMAImaMetric ( float  *pf IDistance,  IMAGE  *pIMA,  IMAGE  *pIMB,  int  inMetricType,  float 
flParam) ; 

int  inIMAImaShift( IMAGE  *pIMImaOut/  IMAGE  *pIMImaIn) ; 

int  inIMASimMap (IMAGE  *pIMMap,  IMAGE  *pIMRef,  IMAGE  *pIMIma,  int  inBlockSize,  int  inMetricType # 
float  flParam) ; 

int  inIMAAverageSimilarity (float  *pflAveSim,  int  inLimit,  IMAGE  *pIMIdeal,  IMAGE  *pIMImage) ; 
int  inlMAZeroPadlmage ( IMAGE  *pIMOutIma/  IMAGE  *pIMInIma,  int  inType); 
int  inIMAGetWindow( IMAGE  *pIMWindow,  int  inWindowType ) ; 

int  inIMALikelihood (float  *pflLik,  IMAGE  *pIMg,  IMAGE  *pIMEstBlur,  IMAGE  *pIMf, 

IMAGE  *pIMPrior/  int  inType) ; 
int  inIMATotalFlip( IMAGE  *pIMOutIma/  IMAGE  *pIMInIma) ; 

- - 


#def ine  FIMAGE  0 
#endif 

// - 
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//  DESCRIPTION  :  Functions  for  manipulating  images 
//  AUTHOR  :  Pablo  Zegers 

//  DATE  :  March,  1999 

// - 


// 


//  INCLUDE  FILES 
// - 


#include  <stdlib.h> 
#include  <stdio.h> 
#include  <io.h> 
#include  <fcntl.h> 
#include  <sys/stat.h> 
#include  <string.h> 
#include  <malloc.h> 

# include  <math.h> 

# include  " Image. h" 

// - 


//  CONSTANTS 
/! - 


#def ine  ENERGYTHRESHOLD  ( float ) 0 . 000001 

II - 


//  PRIVATE  PROTOTYPES 
// - 


IMAGE  *pIMAskImaMem(int  inNumRows,  int  inNumCols)  ; 
int  inFreeImaMem( IMAGE  *pIMIma) ; 

int  inlmaMin ( float  *pflMin,  IMAGE  *pIMIma,  int  inType) ; 
int  inImaMax{ float  *pflMax,  IMAGE  *pIMIma,  int  inType); 
int  inlmaMean (mcomplex  *pmcMean,  IMAGE  *pIMIma) ; 
int  inlmaVar (mcomplex  *pmcVar,  IMAGE  *pIMIma) ; 

int  inlnsertlma {IMAGE  *pIMIma,  IMAGE  *pIMSub,  int  inRowIn,  int  inColIn) ; 

int  inlsWithinScope (IMAGE  *pIMIma,  int  inRow,  int  inCol) ; 

int  inExtractlma { IMAGE  *pIMSub,  IMAGE  *pIMIma,  int  inRowIn,  int  inColIn, 

int  inDelRows,  int  inDelCols); 
int  inCopylma { IMAGE  *pIMDesIma,  IMAGE  *pIMOriIma) ; 
int  inHorFlipIma { IMAGE  *pIMOutIma,  IMAGE  *pIMInIma) ; 
int  inVerFlipIma{ IMAGE  *pIMOutIma,  IMAGE  *pIMInIma) ; 
int  inlmaScaOp ( IMAGE  *pIMImaB,  IMAGE  *pIMImaA,  mcomplex  mcValue, 

int  inOpType) ; 

int  inlmaMa top (IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inOpType); 

int  inlmaFunOp { IMAGE  *pIMImaB,  IMAGE  *pIMImaA,  int  inOpType); 

int  inDFTIma { IMAGE  *pIMOutIma,  IMAGE  *pIMInIma,  int  inTransf ormType) ; 

int  inlmaConvolution ( IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inConType) ; 

int  inlmaCorrelation { IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inCorType) ; 

int  inOTFFromAperture (IMAGE  *pIMOTF,  IMAGE  *pIMAperture ,  int  inCorTyp) ; 

int  inlmaMetric (float  *pf IDistance,  IMAGE  *pIMA,  IMAGE  *pIMB,  int  inMetricType,  float  flParam) ; 
int  inlmaShift (IMAGE  *pIMImaOut,  IMAGE  *pIMImaIn) ; 

int  inSimMap ( IMAGE  *pIMMap,  IMAGE  *pIMRef,  IMAGE  *pIMIma,  int  inBlockSize ,  int  inMetricType, 
float  f lParam) ; 

int  inGetAveSim ( float  *pflAveSim,  IMAGE  *pIMSimMap,  float  flRadius,  float  flWidth, 

IMAGE  *pIMSpectrum) ; 

int  inZeroPadlmage (IMAGE  *pIMOutIma,  IMAGE  *pIMInIma,  int  inType); 
int  inGetWindow{ IMAGE  *pIMWindow,  int  inWindowType ) ; 

int  inLikelihood{ float  *pflLik,  IMAGE  *pIMg,  IMAGE  *pIMEstBlur,  IMAGE  *pIMf, 

IMAGE  *pIMPrior,  int  inType) ; 
float  flLogNatFac { float  flNumber) ; 

int  inTotalFlip( IMAGE  *pIMOutIma,  IMAGE  *pIMInIma) ; 
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// 


//  PUBLIC  ROUTINES 
// - 


// 


// 


IMAGE  *pIMIMAAskImaMem ( int  inNumRows,  int  inNumCols) 

{ 

return (pIMAskImaMem( inNumRows, inNumCols) ) ? 

} 

// - 


// 


int  inIMAFreelmaMem ( IMAGE  *pIMIma) 

{ 

return {inFreelmaMem (pIMIma) ) ; 

> 

// - 


// 


int  inIMASe tlma Index ( IMAGE  *pIMIma,  int  inindex) 

{ 

pIMIma -> inlndex  =  inindex; 
return (OK) ; 

) 

II - 


// 


int  inlMAGetlmalndextint  *pinlndex,  IMAGE  *pIMIma) 

{ 

(♦pinlndex)  =  pIMIma->inIndex; 
return < OK) ; 

} 

// - 


// 


int  inlMASetlmaElem ( IMAGE  *pIMIma,  int  inRow,  int  inCol,  mcomplex  mcValue) 

{ 

int  inStatus; 

inStatus  =  OK; 

pIMIma- >ppmcElem [ inRow] [inCol] .flRe  =  mcValue.flRe; 
pIMIma- >ppmcElem[ inRow] [inCol] . film  =  mcValue.fllm; 

return (inStatus) ? 

} 

// - 


// 


int  inlMAGetlmaElem  (mcomplex  *pmcValue,  IMAGE  *pIMIma,  int  inRow,  int  inCol) 

{ 

int  inStatus; 

inStatus  =  OK; 

pmcValue->f lRe  =  pIMIma ->ppmcElem [inRow] [inCol] .flRe; 
pmcValue->f 11m  =  pIMIma- >ppmcElem[ inRow] [inCol] .film; 
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return {inStatus ) ; 


// 


IMAGE  *pIMIMAReadIma (char  *szName) 

{ 

int  inHandle, 

inindex, 
inNumRows , 
inNumCols , 
inCountl , 
inCount 2 ; 

IMAGE  *pIMIma ; 

inHandle  =  _open (szName , _0_B INARY) ; 

if  (inHandle  <  0) 

{ 

pIMIma  =  NULL; 

} 

else 

{ 

read (inHandle, (void  *) fcinlndex, sizeof (int) ) ; 
read (inHandle, (void  *)& inNumRows , sizeof ( int) ) ; 
read (inHandle, (void  *) ^inNumCols , sizeof (int) ) ; 

pIMIma  =  pIMAsklmaMem (inNumRows,  inNumCols)  ; 

if  (pIMIma  !=  NULL) 

{ 

pIMIma-> inindex  =  inindex; 

for  ( inCount 1=0 ; inCountl<pIMIma->inNumRows ; inCountl++) 

{ 

for  (inCount2=0; inCount2<pIMIma-> inNumCols ; inCount2++) 

{ 

read ( inHandle, (void  *)& (pIMIma->ppmcElem [ inCountl] [inCount2] . flRe) , sizeof ( float) ) ; 
read ( inHandle, (void  *)& (pIMIma->ppmcElem[ inCountl] [inCount2] . film) , sizeof (float) ) ; 
> 

> 

} 

_c lose (inHandle) ; 

} 

return (pIMIma) ; 


// 


int  inIMASavelma (char  *szName,  IMAGE  *pIMIma) 

{ 

int  inStatus, 

inHandle, 
inCountl , 
inCount2 ; 

inStatus  =  OK; 

inHandle  =  _open  (szName , _0_CREAT  | _0_WR0NLY  | __0_B INARY, _S_IREAD |  __S_I WRITE)  ; 

if  (inHandle  <  0) 

/ 

inStatus  =  NK; 

} 

else 

{ 

write (inHandle, (void  *)&(pIMIma->inIndex),sizeof(int)); 
write (inHandle, (void  *)& (pIMIma-> inNumRows) , sizeof (int) ) ; 
write (inHandle, (void  *)& (pIMIma-> inNumCols) , sizeof (int) ) ; 
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for  (inCountl  =  0 ;  inCount l<pIMIma->inNumRows ;  inCountl++) 

for  (inCount 2=0; inCount2<pIMIma->inNumCols ; inCount2++) 

write (inHandle, (void  *) & {pIMIma ->ppmc El  em [inCount 1] [inCount2] -flRe) , sizeof ( float) ) ; 
write ( inHandle, (void  *)& (pIMIma->ppmcElem [ inCount 1] [inCount2] . film) , sizeof (float) ) ; 
) 

} 

_close ( inHandle) ; 

} 

return (inStatus) ; 

} 

// - 


// - 

int  inIMAImaMin ( float  *pflMin,  IMAGE  *pIMIma,  int  inType) 
return ( inImaMin(pflMin,  pIMIma, inType) ) ; 

> 

- - 


- - 

int  inlMAImaMax  ( float  *pflMax,  IMAGE  *pIMIma,  int  inType) 
return (  inlmaMax (pf  lMaxr  pIMIma ,  inType)  )  ; 

} 

- - 


- - 

int  inIMAImaMean (mcomplex  *pmcMean,  IMAGE  *pIMIma) 

{ 

return  ( inlmaMean (pmcMean,  pIMIma )  )  ; 

} 

- - 

- 

int  inIMAImaVar (mcomplex  *pmcVar,  IMAGE  *pIMIma) 

{ 

r e turn (inlmaVar (pmcVar , pIMIma) )  ; 

} 

- - 


- - 

int  inlMAInsertlma ( IMAGE  *pIMIma,  IMAGE  *pIMSub,  int  inRowIn,  int  inCoIIn) 
return  ( inlnsertlma  (pIMIma # pIMSub,  inRowIn,  inColIn)  )  ; 

} 

- - 


- -  - 

int  inlMAExtractlma (IMAGE  *pIMSub,  IMAGE  *pIMIma/  int  inRowIn,  int  inColIn,  int  inDelRows,  int 
inDelCols) 

return ( inEx tract Ima  (pIMSub, pIMIma,  inRowIn,  inColIn,  inDelRows ,  inDelCols) )  ; 

} 

- - 


- - 

int  inIMACopylma ( IMAGE  *pIMDesIma,  IMAGE  *pIMOriIma) 

{ 
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return  ( inCopy Ima  (pIMDesIma,  pIMOrilma)  )  ; 


} 

// 


// - 

int  inIMAHorFlipIma { IMAGE  *pIMOutIma,  IMAGE  *pIMInIma) 

{ 

return ( inHorFlipIma (pIMOutlma, pIMInlma) ) ; 

} 

// - 


// 


int  inIMAVerF lip Ima (IMAGE  *pIMOutIma,  IMAGE  *pIMInIma) 

{ 

return { inVerFlipIma (pIMOutlma, pIMInlma) ) ; 

} 

II - - - 


// 


int  inIMARes ample (IMAGE  *pIMOutIma,  IMAGE  *pIMInIma) 

{ 

int  inStatus, 

inFirSeg, 

inSecSeg; 

IMAGE  *pIMBufferl, 

*pIMBuf fer2 , 

*pIMBuf fer3  ; 


inStatus  =  OK; 

if  (  (pIMOutlma ->inNumRows  !=  pIMInIma->inNumRows )  &&  (pIMOutIma-> inNumCo Is  !=  pIMInlma- 
>inNumCols) ) 

{ 

//  Gets  the  spectrum  of  the  incoming  image 

pIMBufferl  =  pIMIMAAsklmaMem (pIMInlma- > inNumRows ,pIMInIma->inNumCols)  ; 
inDFTIma (pIMBufferl , pIMInlma , DFT ) ? 

//  Row  resampling 

pIMBuffer2  =  pIMIMAAsklmaMem ( pIMOutlma- > inNumRows , pIMBuff erl->inNumCols ) ; 
if  (pIMBuff er 2 -> inNumRows  <  pIMBuf ferl-> inNumRows ) 

{ 

inFirSeg  =  (( int ) ceil ((( float) pIMBuf fer 2 -> inNumRows) /2 )) ; 
inSecSeg  =  pIMBuf f er2->inNumRows- inFirSeg; 

pIMBuffer3  =  pIMAsklmaMem (inFirSeg, pIMBuf fer2->inNumCols) ; 

inExtractlma (pIMBuf fer3 , pIMBuf ferl, 0, 0, inFirSeg, pIMBuf fer2->inNumCols) ; 
inlnsertlma (pIMBuf fer2 , pIMBuf fer3 ,0,0); 
inFreelmaMem (pIMBuf fer3 ) ; 

pIMBuf fer 3  =  pIMAsklmaMem { inSecSeg, pIMBuf fer2->inNumCols ) ; 

inExtractlma (pIMBuf fer3 , pIMBuf ferl , pIMBuf ferl -> inNumRows -inSecSeg, 0 , inSecSeg, pIMBuf fer 2- 
>inNumCols ) ; 

inlnsertlma (pIMBuf fer2 , pIMBuf fer3 , inFirSeg,  0) ; 
inFreelmaMem (pIMBuf fer3 ) ; 

> 

else 

{ 

inZeroPadlmage (pIMBuf f er2 , pIMBuf ferl, SPREAD) ; 

} 

//  frees  the  memory  used  to  store  the  original  spectrum 
inFreelmaMem (pIMBufferl) ; 

//  Column  Resampling 

pIMBufferl  =  pIMIMAAsklmaMem (pIMOutIma-> inNumRows ,pIMOutIma-> inNumCo Is)  ; 
if  (pIMBuf ferl->inNumCols  <  pIMBuf fer2-> inNumCo Is ) 

{ 

inFirSeg  =  (( int) ceil ((( float)pIMBuf ferl->inNumCols) /2 )) ; 
inSecSeg  =  pIMBuf ferl->inNumCols-inFirSeg; 
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pIMBuf f er3  =  pIMAsklmaMem  (pIMBuf  ferl->inNumRows ,  inFirSeg)  ; 

inExtractlma  (pIMBuf  fer3  ,pIMBuf  fer2 , 0, 0 , pIMBuf ferl->inNumRows ,  mFirSeg)  ; 
inlnsertlina  (pIMBuf  ferl ,  pIMBuf  fer3  ,  0 , 0)  ; 
inFreelmaMem (pIMBuf fer3 )  ; 

pIMBuf  fer3  =  pIMAsklmaMem  (pIMBuf  f erl->inNumRows ,  inSecSeg)  ; 

inExtractlma (pIMBuf fer3 , pIMBuf f er2 , 0 , pIMBuf f er2 ->inNumCols-inSecSeg/ pIMBuf f  erl- 
>inNumRows, inSecSeg) ; 

inlnsertlma (pIMBuf ferl , pIMBuf fer3 ,0, inSecSeg) ; 
inFreelmaMem (pIMBuf fer3 ) ; 

} 

else 

inZeroPadlmage (pIMBuf ferl , pIMBuf fer2  , SPREAD)  ; 

} 

//  frees  the  memory  used  to  store  the  intermediate  spectrum 

inFreelmaMem (pIMBuf fer2 ) ; 

inDFTIma (pIMOutlma, pIMBuf f erl, IDFT) ; 

//  frees  the  memory  used  to  store  the  final  spectrum 
inFreelmaMem  (pIMBuf  ferl)  ; 

} 

else 

inCopylma  (pIMOutlma,  pIMInlma)  ; 

} 

return (inStatus) ; 

} 

- - 


it - 

int  inIMAImaScaOp ( IMAGE  *pIMImaB#  IMAGE  *pIMImaA,  mcomplex  mcValue,  int  inOpType) 

^  return ( inlmaScaOp (pIMImaB, pIMImaA, mcValue, inOpType) ) ; 

} 

- - 


- - 

int  inIMAImaMatOp ( IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inOpType) 

return  (inlmaMatOp  (pIMImaC,  pIMImaA,pIMImaB,  inOpType)  )  ; 

} 

it - 


it - 

int  inIMAImaFunOp ( IMAGE  *pIMImaB,  IMAGE  *pIMImaA/  int  inOpType) 

{  return ( inlmaFunOp (pIMImaB, pIMImaA, inOpType) ) ; 

} 

- - 


a - 

int  inIMADFTIma ( IMAGE  *pIMOutIma,  IMAGE  *pIMInIma,  int  inTransf ormType) 

{  return (inDFTImatpIMOutlma^pIMInlma, inTransf ormType) ) ; 

} 

- - 


- - 

int  inIMACLS( IMAGE  *pIMf,  IMAGE  *pIMg,  IMAGE  *pIMh,  float  flGamma,  int  inType) 

{ 

inStatus, 

inCount 1, 
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float 


mcomplex 

IMAGE 


inCount2 , 

inXCenter, 

inYCenter; 

f lMagl , 

f lMag2 , 
flBuffer; 
mcValuel, 

mcValue2 ; 

*pIMBuf ferl , 

*pIMBuffer2, 
*pIMBuf f er3 ; 


inStatus  =  OK; 

pIMBufferl  =  p IMAsklmaMem (pIMf ->inNumRows, pIMf- >inNumCols)  ; 
inDFTIma (pIMBufferl , pIMBufferl , DFT) ; 

pIMBuf  f  er2  =  pIMAskImaMem{pIMf->inNumRows,pIMf->inNumCols); 
voSetCom (tmcValuel ,0,0)  ; 

for  (inCount 1=0; inCount l<pIMf ->inNumRows; inCount 1++) 
for  (inCount2=0;inCount2<pIMf->inNumCols; inCount2++) 

voAssignCom ( Sc (pIMBuf fer 2 ->ppmcElem [inCount 1] [inCount2] ) ,mcValuel) ; 

inXCenter  =  pIMf ->inNumRows/2 ; 
inYCenter  =  pIMf ->inNumCols/2 ; 

voSetCom (&mcValuel , +1.00,0) ; 
voSetCom { &mcValue2 ,-0.25,0); 

voAs s ignCom( & (pIMBuf fer 2 ->ppmcElem[ inXCenter 3  [inYCenter]) ,mcValuel) ; 
voAssignCom (& (pIMBuf fer2->ppmcElem[ inXCenter- 1) [inYCenter] ) ,mcValuel) ; 
voAssignCom (& (pIMBuf fer2->ppmcElem [inXCenter +1] [inYCenter] ) ,mcValuel) ; 
voAs signCom( Sc  (pIMBuf  fer2->ppmcElem[ inXCenter]  [inYCenter-1]  )  ,mcValuel)  ; 
voAssignCom (& (pIMBuf f er2->ppmcElem [ inXCenter] [ inYCenter+1 ] ) ,mcValuel) ; 

pIMBuffer3  =  pIMAsklmaMem (pIMf ->inNumRows , pIMf ->inNumCols )  ; 

inDFTIma (pIMBuf  fer 3 , pIMBuf fer2 , DFT) ; 

pIMBuffer3  =  pIMAs klmaMem (pIMf ->inNumRows, pIMf ->inNumCols ) ; 

for  ( inCountl=0; inCountl<pIMf->inNumRows; inCount 1++) 

{ 

for  (inCount2=0;inCount2<pIMf->inNumCols; inCount2++) 

{ 

voMagCom(&f lMagl, pIMBuf ferl ->ppmcElem[ inCountl] [inCount 2] ) ; 
voMagCom(&flMag2 , pIMBuf fer3->ppmcElem[ inCount 1] [inCount2] ) ; 
f IBuffer  =  f lMagl *f lMagl +f lGamma*f lMag2 *flMag2 ; 
voSetCom (&mcValuel , f IBuffer,  0) ; 

voCon  jugateCom  ( ScmcValue2 ,  pIMBuf  fer  l->ppmcElem  [  inCountl  ]  [inCount2J )  ; 
voDivCom (& (pIMBuf fer2->ppmcElem[ inCount 1] [inCount2] ) ,mcValue2 ,mcValuel) ; 
) 

} 

inDFTIma (pIMBufferl , pIMBuf f er2 , IDFT) ; 
inlmaConvolution (pIMBuf f er2 , pIMBufferl , pIMg, inType) ; 
inIMAImaFunOp (pIMf , pIMBuf fer2, IMRA) ; 

inFreelmaMem (pIMBuf ferl ) ; 
inFreelmaMem (pIMBuf fer2 ) ; 
inFreelmaMem (pIMBuf fer3 ) ; 

return ( inStatus) ; 


// 


int  inIMAMaxLik( IMAGE  *pIMf,  IMAGE  *pIMg,  IMAGE  *pIMh,  IMAGE 

int  inType) 


{ 

int 


inStatus, 


*pIMPrior, 


int  iriNumlter, 
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IMAGE 


inCount 1  ; 


*pIMBuf ferl , 

*pIMBuf fer2 ; 


inStatus  =  OK; 

pIMBufferl  =  pIMAsklmaMem (pIMf->inNumRows ,  pIMf->inNumCols )  ; 
pIMBuf  f er2  =  pIMAsklmaMem {pIMf ->iriNumRows , pIMf ->inNumCols )  ; 

inlMACopylma (pIMf , pIMPrior ) ; 

for  (inCount 1=0;  inCount 1< inNumlter;  inCountl++) 

{ 

inlmaConvolution (pIMBuf ferl, pIMf , pIMh, inType) ; 
inlmaMatOp (pIMBuf fer2 , pIMg, pIMBuf ferl , DIV) ; 
inlmaConvolution (pIMBufferl, pIMBuf fer2 ,pIMh, inType) ; 

inCopylma (pIMBuf fer2, pIMf ) ; 
inlmaMatOp (pIMf , pIMBuf fer2 , pIMBuf ferl , MUL) ; 
inCopylma (pIMBuf fer2 , pIMf ) ; 
inlmaFunOp (pIMf , pIMBuf fer2, IMRA) ; 


inFreelmaMem (pIMBufferl) ; 
inFreelmaMem (pIMBuf fer2 ) ; 

return ( inStatus)  ; 


int  inIMAMAP ( IMAGE  *pIMf,  IMAGE  *pIMg,  IMAGE  *pIMh,  IMAGE  *pIMPrior,  int  inNumlter, 

int  inType) 

{ 

int  inStatus, 

inCount 1 ; 

me  omp lex  me Va lue ; 

IMAGE  *pIMBufferl, 

*pIMBuf f er2 ; 


inStatus  =  OK; 

pIMBufferl  =  pIMAsklmaMem (pIMf->inNumRows, pIMf ->inNumCols ) ; 
pIMBuf fer2  =  pIMAskImaMem(pIMf->inNumRows,pIMf->inNumCols); 

inlMACopylma (pIMf , pIMPrior) ; 

for  (inCount 1=0; inCountl< inNumlter ; inCountl++) 

{ 

inlmaConvolution (pIMBuf ferl, pIMf ,pIMh, inType) ; 
inlmaMatOp (pIMBuf fer2 ,pIMg, pIMBufferl, DIV) ; 
voSetCom ( &mcValue ,1,0); 

inlmaScaOp (pIMBuf ferl, pIMBuf fer2,mcValue, SUB) ; 

/ /  CACA 

//  inlmaCorrelation (pIMBuf fer2 , pIMBufferl , pIMh, inType) ; 

inlmaConvolution (pIMBuf fer2 , pIMBuf ferl, pIMh, inType) ; 
inlmaFunOp (pIMBuf ferl, pIMBuf fer2 , EXP) ; 
inCopylma (pIMBuf fer2 , pIMf) ; 

inlmaMatOp (pIMf , pIMBufferl , pIMBuf fer2 , MUL) ; 
inCopylma (pIMBuf ferl, pIMf ) ; 
inlmaFunOp (pIMf, pIMBufferl, IMRA) ; 

) 

inFreelmaMem (pIMBuf ferl) ; 
inFreelmaMem ( pIMBuf fer2) ; 

return (inStatus) ; 

} 

// - 


// - 

int  inIMAElipsoidalAperture (IMAGE  *pIMAperture,  int  inAxisl,  int  inAxis2) 

{ 
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int  inStatus, 

inHalfRow, 
inHalfCol , 
inCountl, 
inCount 2 ; 

float  flXCen, 

f lYCen, 
f IRadius; 


inStatus  =  OK; 

inHalfRow  =  pIMAperture->inNumRows/2 ; 
inHalfCol  =  pIMAperture->inNumCols/2  ; 

for  ( inCount 1=0 ; inCount l<pIMAperture->inNumRows ; inCountl++) 

for  (inCount2=0 ; inCount2<pIMAperture->inNumCols; inCount2++) 

{ 

flXCen  =  (( float) { inCount 1- inHalfRow) ) ? 
f lYCen  =  ( (float) ( inCount2 -inHalfCol) ) ; 
f IRadius  = 

(flXCen* flXCen) / ( ( (float) inAxisl) * ( (float) inAxisl) ) + (f lYCen* f lYCen) / ( ( (float) inAxis2) * ( (float) inA 
xis2)  )  ; 

if  (f IRadius  <  1) 

pIMAperture->ppmcElem[ inCountl) [inCount2 ] . f iRe  =  1; 

pIMAperture->ppmcElem[inCountl] [inCount2] .film  =  0; 

) 

else 

pIMAperture->ppmcElem [ inCountl] [ inCount2 3 . f IRe  =  0; 
p IMAper ture ->ppmc El em[ inCount 1 ] [ inCount2 ] . f 11m  =  0; 

} 

} 

} 

return (inStatus) ; 

) 

- - 


- - 

int  inIMASlantedStripAperture (IMAGE  ^pIMAperture,  int  inLength,  int  inWidth,  float  flAngle) 

{ 

int  inStatus, 

inRow2 , 
inCol2 , 
inLength2 , 
inWidth2 , 
inCountl , 
inCount2 , 
inXCen, 
inYCen, 
inRadius ; 


inStatus  =  OK; 

inRow2  =  (int) (pIMAperture->inNumRows/2) ; 
inCol2  =  (int) (pIMAperture->inNumCols/2) ; 

inLength2  =  (int) (inLength/2) ; 
inWidth2  =  (int) (inWidth/2) ; 

for  (inCount 1=0; inCountl<pIMAperture->inNumRows ; inCountl++) 

for  ( inCount2 =0 ; inCount2<pIMAperture->inNumCols ; inCount2 ++ ) 

{ 

inXCen  =  inCount 1 - inRow2 ; 
inYCen  =  inCount2-inCol2 ; 

inRadius  =  (int) sqrt (inXCen* inXCen+ inYCen* inYCen ) ; 

if  ( (inXCen*cos (flAngle) +inYCen* sin (flAngle ) -inLength2  <  0)  && 

(inXCen*cos (PI+ flAngle ) +inYCen* sin (PI + flAngle ) -inLength2  <  0)  && 
(inXCen*cos (PI/2+f lAngle) +inYCen*sin (PI/2+f lAngle) -inWidth2  <  0)  && 
(inXCen*cos (3*PI/2+f lAngle) +inYCen*sin (3 *PI/2+ flAngle) -inWidth2  <  0) ) 


171 


pIMAperture->ppmcElem [inCountl 3  [ inCount2 ). flRe  =  1; 

pIMAperture->ppmcElem [ inCountl ] [inCount2] .film  =  0; 

> 

else 

pIMAperture->ppmcElem [ inCountl 3 [inCount2 3 . flRe  =  0; 
pIMAperture->ppmcElem[ inCountl 3  [inCount2 3  •  film  =  0; 

} 

} 

} 

return (inStatus) ; 

} 

// - 


// 


int  inIMAImaConvolution ( IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inConType) 
return ( inlmaConvolution (pIMImaC,pIMImaA, pIMImaB, inConType)  )  ; 

} 

- - 

// - 

int  inIMAImaCorrelation ( IMAGE  *pIMImaC,  IMAGE  *pIMImaA/  IMAGE  *pIMImaB/  int  inCorType) 
return (inlmaCorrelation (pIMImaC, pIMImaA, pIMImaB, inCorType) ) ; 

} 

- - 

- - 

int  inlMAOTFFromAperture { IMAGE  *pIMOTF/  IMAGE  *pIMAperture ,  int  inConType) 

{  return ( inOTFFromAperture (pIMOTF, pIMAperture, inConType) ) ; 

} 

- - 

- - 


int  inIMAPSFFromAperture { IMAGE  ♦pIMPSF,  IMAGE  *pIMAperture #  int  inConType) 

{ 

int  inStatus; 

IMAGE  *pIMBufferl, 

*pIMBuffer2; 


inStatus  =  OK; 

pIMBuf  ferl  =  pIMAsklmaMem  (pIMAperture->inNumRows ,  pIMAperture->inNumCols)  ; 
pIMBuf  f er2  =  pIMAsklmaMem (pIMAperture->inNumRows , pIMAperture~>inNumCols )  ; 

inOTFFromAperture  (pIMBuf  ferl  ,pIMAperture,  inConType)  ; 
inDFTIma (pIMPSF, pIMBuf ferl , IDFT) ; 

inFreeImaMem{ pIMBuf ferl) ; 
inFreelmaMem (pIMBuf fer2 ) ; 

return ( inStatus) ; 

} 

- - 


- - 

int  inIMAStretch( IMAGE  *pIMOutIma/  IMAGE  *pIMInIma,  int  inMinVal,  int  inMaxVal) 
{ 

int  inStatus, 

inCountl , 
inCount 2 ; 

float  doMin# 
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doMax , 
f lBuf ferl, 
f lBuf fer2 ; 

mcomplex  mcValuel, 

mcValue2 ; 

inStatus  =  OK; 

inImaMin(&doMin,pIMInIma7REAL) ; 
inlmaMax ( &doMax, pIMInlma, REAL) ; 

if  (doMin  !=  doMax) 

{  flBufferl  =  { (float) inMaxVal- (float) inMinVal) / (doMax-doMin) ; 
f lBuf f er2  =  -flBufferl*doMin+ ( (float) inMinVal) ; 

for  { inCount 1=0 ; inCount l<pIMInIma->inNumRows ; inCountl++ ) 

*  for  (inCount2=0 ; inCount2<pIMInIma->inNumCols ; inCount2++) 

inlMAGet ImaE 1 em ( kmcValuel , pIMInlma , inCount 1 , inCount 2 ) ; 

voSetCom ( &mcValue2 ,  f lBuf f erl*mcValuel . f IRe+f lBuf fer2 # f lBuf ferl*mcValuel . f llm+f lBuf fer2) ; 
inlMASetlmaElemCpIMOutlma, inCountl , inCount2 #mcValue2 ) ; 

> 

} 

} 

return (inStatus) ; 


// 


- - 

int  inIMAImaMetric{ float  *pdoDistance ,  IMAGE  *pIMA,  IMAGE  *pIMB,  int  inMetricType ,  float  flParam) 
{  return (inlmaMetric (pdoDistance # pIMA, pIMB, inMetricType , flParam) ) ? 


// 


// - 

int  inIMAImaShift (IMAGE  *pIMImaOut,  IMAGE  *pIMImaIn) 
{ 

return ( inlmaShif t (pIMImaOut , pIMImaln) ) ; 

} 

// - 


- - 

int  inIMASimMap (IMAGE  *pIMMap,  IMAGE  *pIMRef,  IMAGE  *pIMIma,  int  inBlockSize,  int  inMetricType, 
float  flParam) 

return ( inSimMap (pIMMap , pIMRef , pIMIma , inBlockSize, inMetricType, flParam) ) ; 


// 


- - 

int  inIMAAverageSimilarity ( float  *pflAveSim,  int  inLimit,  IMAGE  *pIMIdeal,  IMAGE  *pIMImage) 

{ 

int  inStatus, 

inRowCenter , 
inColCenter , 
inCountl , 
inCount2 , 
inCount3 ; 

float  f lRadius , 

flBufferl, 
flMagnitude, 
f 1 Energy ; 

IMAGE  *pIMBuf ferl, 

*pIMBuf f er2 , 
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■pIMBuf  fer3  ; 


inStatus  =  OK; 

pIMBufferl  =  pIMAsklmaMem { pIMIdeal ->inNumRows , pIMIdeal->inNumCols )  ; 
pIMBuf  fer2  =  p  IMAs  klmaMem  ( pIMI  dea  1  -  >  iriNumRows , p IMI dea 1 - >  inNumCo  Is); 
pIMBuf fer3  =  pIMAsklmaMem (pIMIdeal->inNumRows , pIMIdeal->inNumCols ) ; 

inDFTIma (pIMBuf ferl , pIMIdeal , DFT) ; 
inlmaShif t (pIMBuf fer2 , pIMBuf ferl ) ; 

inDFTIma (pIMBuf ferl , pIMImage, DFT) ; 
inlmaShif t (pIMBuf fer3 , pIMBuf ferl) ; 

inSimMap  (pIMBuf  ferl ,  pIMBuf  fer2 ,  pIMBuf  fer3 , 4  #  CORRELATION^)  ; 

inRowCenter  =  ( int) ( floor (pIMIdeal ->inNumRows/ 2 )) ; 

inColCenter  =  ( int) ( floor (pIMIdeal ->inNumCo Is/ 2 )) ; 

for  { inCount 1=0; inCount l<inLimit; inCount 1++) 

{ 

pflAveSim[inCountl]  =  0; 
flBufferl  =  0; 

for  { inCount 2=0 ; inCount2<pIMIdeal->inNumRows ; inCount2++) 

{ 

for  ( inCount3=0 ; inCount3<pIMIdeal->inNumCols ; inCount3++) 

flRadius  =  (float) sqrt ( (inCount2- inRowCenter ) * (inCount2- inRowCenter ) + ( inCount3- 
inColCenter ) * (inCount3-inColCenter) ) ; 

if  ({flRadius  <  inCountl+5)  &&  (inCountl-5  <  flRadius)) 

{ 

voMagCom (kflMagnitude, pIMBuf fer2->ppmcElem[inCount2 ] [inCount3] ) ; 
fl Energy  =  flMagnitude*flMagnitude; 

pflAveSim[ inCount 1]  +=  ( f lEnergy*pIMBuf ferl->ppmcElem[ inCount2 ] [inCount3 ] . f IRe) ; 
flBufferl  +=  f 1 Energy ; 

} 

> 

} 

pf lAveSim[ inCount 1]  /=  flBufferl; 

} 

inFreelmaMem (pIMBuf ferl) ; 
inFreelmaMem (pIMBuf fer2 ) ; 
inFreelmaMem (pIMBuf fer3) ; 

return ( inStatus) ; 


// 


int  inlMAZeroPadlmage ( IMAGE  *pIMOutIma/  IMAGE  *pIMInIma/  int  inType) 

{ 

return ( inZeroPadlmage (pIMOutlma, pIMInlma, inType) ) ; 

} 

n - 


II - - 

int  inIMAGetWindow{  IMAGE  *pIMWindow,  int  inWindowType) 

{ 

return (inGetWindow(pIMWindow, inWindowType) ) ; 

} 

// - 


// 


int  inIMALikelihood( float  *pflLik,  IMAGE  *pIMg,  IMAGE  *pIMEstBlur,  IMAGE  *pIMf, 

IMAGE  *pIMPrior,  int  inType) 


{ 

return ( inLikelihood (pf iLik,  pIMg,  pIMEstBlur , pIMf  #  pIMPrior , inType) ) ? 

} 
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// 


// - 

int  inIMATotalFlip( IMAGE  *pIMOutIma,  IMAGE  *pIMInIma) 
return ( inTotalFlip (pIMOutlma , pIMInlma) ) ; 

} 

// - 

//  PRIVATE  ROUTINES 

- - 


// - 

// - 


IMAGE  *pIMAskImaMem  ( int  inNumRows,  int  inNumCols) 

{ 

int  inStatus, 

inCount 1 ; 

IMAGE  *pIMIma; 

inStatus  =  OK; 

pIMIma  =  {IMAGE  *}  calloc { 1 , sizeof ( IMAGE) ) ; 

if  (pIMIma  ==  NULL) 

{ 

inStatus  =  NK; 

} 

else 

{ 

pIMIma ->inlndex  =  0; 

pIMIma- >inNumRows  =  inNumRows; 
pIMIma-> inNumCols  =  inNumCols? 

pIMIma ->ppmcElem  =  (mcomplex  **)  calloc ( inNumRows, sizeof (mcomplex  *)); 
if  (pIMIma~>ppmcElem  ==  NULL) 

{ 

inStatus  -  NK; 

} 

else 

{ 

for  { inCount 1=0 ; inCount 1< inNumRows ; inCountl++) 

{ 

pIMIma ->ppmcElem[ inCount 13  =  (mcomplex  *)  calloc ( inNumCols , sizeof (mcomplex) ) ; 
if  (pIMIma->ppmcElem[inCountl]  ==  NULL) 

{ 

inStatus  =  NK; 

} 

} 

} 

) 

if  (inStatus  ==  NK) 

inFreelmaMem (pIMIma) ; 

return (pIMIma) ; 

} 

- - 


// 


int  inFreelmaMem { IMAGE  *pIMIma) 

{ 

int  inStatus, 

inCount 1 ; 


inStatus  =  OK; 
if  (pIMIma  ! =  NULL) 
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{ 


if  (pIMIma->ppmcElem  !=  NULL) 

for  ( inCount 1=0 ; inCount  l<p  IMIma ->  inNumRows ; inCountl++) 
if  (pIMIma->ppmcElem[ inCountl )  !=  NULL) 

free (pIMIma->ppmcElem[ inCount 1] ) ; 


free (pIMIma->ppmcElem) ; 
} 


free (pIMIma) ; 

} 

return (inStatus) ; 


// 


// 


int  inlmaMin ( float  *pflMin, 
{ 

int 


float 

inStatus  =  OK; 

switch  (inType) 

{ 

case  REAL: 

(*pflMin)  =  pIMIma->ppmcElem [0] [0] . flRe; 

for  (inCount 1=0; inCount 1 <p IMIma -> inNumRows ; inCountl++) 

{ 

for  ( inCount2=0 ; inCount2<pIMIma->inNumCols ; inCount2++) 

{ 

flBuffer  =  pIMIma->ppmcElem [inCountl] [inCount2] . flRe; 
if  (flBuffer  <  (*pflMin)) 

( *pf lMin)  =  flBuffer; 

> 

} 

break; 

case  IMAGINARY: 

( *pf lMin)  =  p IMIma - >ppmc E 1 em [ 0 ] [0] .film; 

for  ( inCountl=0 ; inCount l<p IMIma- > inNumRows ; inCountl++) 

{ 

for  (inCount2=0; inCount2<pIMIma->inNumCols ; inCount2++) 

{ 

flBuffer  =  pIMIma->ppmcElem[inCountl] [inCount2] . film; 
if  (flBuffer  <  (*pflMin)) 

( *pf lMin)  =  flBuffer; 

} 

} 

break; 

case  MAGNITUDE: 

voMagCom(pf lMin, pIMIma- >ppmcElem[0] [0] ) ; 

for  ( inCountl=0; inCountl<pIMIma-> inNumRows; inCountl++) 

{ 

for  (inCount2=0; inCount2<pIMIma->inNumCols ; inCount2++) 

{ 

voMagCom(&f lBuf f er ,p IMIma ->ppmcElem{ inCount 1] [inCount 2 3 ) ; 
if  (flBuffer  <  (*pflMin)) 

( *pf lMin)  =  flBuffer; 

> 

} 

break; 

} 

return (inStatus) ; 


// 


IMAGE  *pIMIma,  int  inType) 

inStatus , 

inCountl , 
inCount2 ; 
flBuffer; 
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//— 


int  inImaMax( float  *pflMax,  IMAGE  *pIMIma,  int  inType) 

{ 

int  inStatus, 

inCountl , 
inCount 2 ; 

float  flBuffer; 

inStatus  =  OK; 

switch  (inType) 

{ 

case  REAL: 

( *pf IMax)  =  pIMIma->ppmcElem[0] [0] . f IRe; 

for  ( inCountl=0 ; inCountl<pIMIma->inNumRows ; inCountl++ ) 

for  ( inCount2=0 ; inCount2<pIMIma->inNumCols ; inCount2++) 

flBuffer  =  pIMIma->ppmcElem [inCountl 3  [inCount2] . flRe; 
if  (flBuffer  >  (*pflMax)) 

(*pflMax)  =  flBuffer; 

} 

} 

break; 

case  IMAGINARY: 

(*pflMax)  =  pIMIma->ppmcElem[0] [0] . film; 

for  (inCount 1=0; inCountl<pIMIma->inNumRows ; inCount 1++) 

for  (inCount2=0; inCount2<pIMIma->inNumCols; inCount2++) 

{ 

flBuffer  =  pIMIma->ppmcElem [ inCountl ] [inCount2] . film; 
if  (flBuffer  >  (*pflMax>) 

{ *pf IMax)  =  flBuffer; 

} 

) 

break; 

case  MAGNITUDE: 

voMagCom ( p f IMax , pIMIma- >ppmcElem [ 0 ]  [0] )  ; 

for  ( inCount 1=0 ; inCount l<pIMIma->inNumRows ; inCount 1++) 

for  (inCount2=0; inCount2<pIMIma->inNumCols ; inCount2++) 

{ 

voMagCom (&f lBuf fer, pIMIma->ppmcElem[ inCountl] [inCount2] ) ; 
if  (flBuffer  >  ( *pf IMax) ) 

( *pf IMax)  =  flBuffer; 

} 

} 

break; 
case  PHASE: 

voPhaCom (pf IMax, pIMIma->ppmcElem [0] [0] ) ; 

for  (inCount 1=0; inCount 1 <pIMIma -> inNumRows ; inCount 1++) 

{ 

for  ( inCount2=0 ; inCount2<pIMIma->inNumCols ; inCount2++) 

{ 

voPhaCom (ScflBuffer/pIMIma->ppmcElem[ inCountl]  [inCount2]  )  ; 
if  (flBuffer  >  (*pflMax)) 

(*pflMax)  =  flBuffer; 

> 

} 

break; 

} 

return (inStatus) ; 

} 

- - 


- - 

int  inlmaMean (mcomplex  *pmcMean,  IMAGE  *pIMIma) 

{ 

int  inStatus, 
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mcomplex 


inCountl, 
inCount 2 ; 

mcBuf ferl, 

mcBuf fer2  ? 


inStatus  =  OK; 
voSetCom (pmcMean, 0, 0) ; 

for  ( inCount  1=0 ;  inCount l<pIMIma->inNumRows ;  inCount 1++) 

for  ( inCount2=0 ; inCount2<pIMIma->inNumCols ; inCount2++) 

{ 

voAs s ignCom (fcmcBuf ferl, *pmcMean) ; 

voAddCom (pmcMean, pIMIma->ppmcElem[ inCountl] [inCount2] , mcBuf ferl) ; 

} 

} 

voAs s ignCom (fcmcBuf ferl, * pmcMean) ; 

voSetCom ( kmcBuf f er2 ,  (float)  (pIMIma->inNumRows*pIMIma->inNumCols )  ,0)  ; 
voDivCom (pmcMean, mcBuf f erl , mcBuf fer2 ) ; 

return (inStatus) ? 


int  inlmaVar (mcomplex 
{ 

int 


mcomplex 


*pmcVar,  IMAGE  *pIMIma) 

inStatus , 

inCountl, 
inCount2 ; 

mcMean, 

mcBuf ferl, 
mcBuf fer2 ; 


inStatus  =  OK; 

inlmaMean ( imcMean , pIMIma ) ; 

voSetCom (pmcVar ,0,0); 

for  (inCountl=0;inCountl<pIMIma->inNumRows; inCountl++) 

for  ( inCount2=0 ; inCount2<pIMIma~>inNumCols ; inCount2++) 

voConjugateCom(&mcBuff erl, pIMIma->ppmcElem[ inCountl]  [inCount2]  )  ; 
voMulCom(&mcBuf fer2 , pIMIma->ppmcElem [ inCountl ] [inCount2] ,mcBufferl) ? 
vo As  s ignCom ( &mcBu  f  f  er 1 , *pmcVar) ; 

voAddCom (pmcVar, mcBuf ferl, mcBuf fer2) ; 

} 

} 

voAss ignCom (fcmcBuf ferl, *pmcVar) ; 

voSetCom (&mcBuffer2  ,  (float)  ( pIMIma ->inNumRows*pIMIma->inNumCols) ,0) ; 
voDivCom (pmcVar , mcBuf ferl , mcBuf fer2 ) ; 

voConjugateCom ( &mcBuf f erl , mcMean) ; 
voMulCom(&mcBuf fer2 , mcMean, mcBuf ferl) ; 
voAs s ignCom (&mcBuf ferl, *pmcVar) ; 
voSubCom ( pmcVar , mcBuf  ferl , mcBuf  fer2 ) ; 

return (inStatus) ; 

} 

f  - - 


- - 

int  inlsWithinScope  (IMAGE  *pIMIma,  int  inRow,  int  inCol) 

{ 

int  inResult ; 

inResult  =  YES; 

if  ((inRow  <  0)  |  |  (pIMIma->inNumRows  <=  inRow)) 
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inResult  =  NO; 


if  ((inCol  <  0)  ||  (pIMIma->iriNumCols  <=  inCol) ) 

inResult  =  NO; 


return (inResult) ; 

} 

// - 


//  — 


int  inlnsertlma (IMAGE 
{ 

int 


*pIMImaf  IMAGE  *pIMSub,  int  inRowIn,  int  inColIn) 

inStatus, 

inRowLim, 
inColLim, 
inCountl , 
inCount 2 ; 


inStatus  =  OK; 

inRowLim  =  inRowIn+pIMSub->inNumRows ; 
inColLim  =  inColIn+pIMSub->inNumCols ; 

for  ( inCount l=inRowIn; inCount 1< inRowLim; inCount 1++) 

{  for  ( inCount 2= inCol In; inCount2<inColLim; inCount2++) 

if  (inlsWithinScope (pIMIma, inCountl, inCount2)  ==  YES) 

if  ( inlsWithinScope (pIMSub, inCountl- inRowIn, inCount2-inColIn)  ==  YES) 

voAssignCom ( & (pIMIma->ppmcElem[ inCountl] [inCount2] ) /pIMSub->ppmcElem [ inCountl - 
inRowIn) [ inCount2-inColIn] ) ; 

else 

voSetCom (& (pIMIma->ppmcElem[ inCountl] [inCount2] ) , 0, 0) ; 

> 

} 

} 

re turn ( inStatus ) ; 

} 


- - 

int  inExtractlma (IMAGE  *pIMSub,  IMAGE  *pIMIma#  int  inRowIn,  int  inColIn,  int  inDelRows,  int 
inDelCols ) 

{ 

int  inStatus, 

inRowLim, 
inColLim, 
inCountl , 
inCount2 ; 


inStatus  =  OK; 

inRowLim  =  inRowIn+ inDelRows ; 
inColLim  =  inColIn+inDelCols ; 

for  (inCountl=inRowIn; inCount 1< inRowLim; inCountl++) 

for  (inCount 2= inCol In; inCount2< inColLim; inCount2++) 

if  ( inlsWithinScope (pIMSub, inCountl -inRowIn, inCount2-inColIn)  ==  YES) 

if  ( inlsWithinScope (pIMIma, inCountl, inCount2)  ==  YES) 

voAssignCom(& (pIMSub- >ppmcElem[ inCountl -inRowIn] [ inCount 2 -inCol In] ) ,pIMIma- 
>ppmcElem[ inCountl] [inCount2] ) ; 
else 

voSetCom (&  (pIMSub- >ppmcElem[ inCountl- inRowIn]  [  inCount2-inColIn] )  ,0,0)  ? 

} 

} 

) 
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return ( inStatus) ; 


} 

int  inCopylma ( IMAGE  *pIMDesIma,  IMAGE  *pIMOriIma) 

{  return ( inExtractlma (pIMDesIma, pIMOrilma ,0,0, pIMOriIma->inNumRows , pIMOriIma->inNumCols ) ) ; 

} 

int  inHorFlipIma (IMAGE  *pIMOutIma,  IMAGE  *pIMInIma) 

*  -inf  inStatus, 

inCountl , 
inCount 2 ; 


inStatus  =  OK; 

for  ( inCountl=0 ? inCountl<pIMInIma->inNumRows  ? inCountl++) 
for  (inCount2=0;inCount2<pIMInIma->inNumCols;inCount2++) 

voAssignCom (& (pIMOutIma->ppmcElem[pIMInIma->inNumRows-inCountl-l ] [xnCount2] ) ,pIMInIma- 
>ppmcElem[ inCountl] [inCount2] ) ; 

return (inStatus) ; 


// - 


// - 


int  inVerFlipIma ( IMAGE 
{ 

int 


*pIMOutIma,  IMAGE  *pIMInIma) 

inStatus , 

inCountl , 
inCount2 ; 


inStatus  =  OK; 

for  ( inCountl=0 ; inCount l<pIMInIma->inNumRows ; inCountl++) 
for  (inCount2=0;inCount2<pIMInIma->inNumCols; inCount2++) 

voAssignCom ( & (pIMOutIma->ppmcElem [ inCountl ] [pIMInIma->inMumCols- inCount 2-1] ) ,pIMInIma- 
>ppmcElem[ inCountl] [inCount2] ) ; 

return (inStatus) ; 

} 

- - 


- - 

int  inImaFunOp( IMAGE  *pIMImaB,  IMAGE  *pIMImaA/  int  inOpType) 

{ 

int 


float 

inStatus  =  OK; 

switch  (inOpType) 

{ 

case  ZERE: 

for  (inCount 1=0; inCount l<pIMImaB->inNumRows ; inCount 1++) 
for  ( inCount2=0 ; inCount2<pIMImaB->inNumCols ; inCount2++) 

voSetCom(& (pIMImaB->ppmcElem{ inCountl] [inCount2] ) ,0,pIMImaA- 
>ppmcElem[ inCountl] [inCount2] .film) ; 
break; 

case  ZEIM: 

for  ( inCount 1=0 ; inCount l<pIMImaB->inNumRows ; inCount 1++) 


inStatus, 

inCountl, 
inCount 2 ; 
flBufferl; 
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for  { inCount 2  =  0 ;  inCount2<pIMImaB->inNumCols ;  inCount2++) 

voSetCom ( & {pXMImaB->ppmcElem[ inCount 1] [inCount2]) ,pIMImaA- 
>ppmcElem [ inCountl 3  [inCount2] . flRe,0) ; 
break; 

case  IMRA: 

for  ( inCount 1=0 ; inCount l<pIMImaB-> inNumRows ; inCountl++) 
for  { inCount2=0; inCount2<pIMImaB->inNumCols ; inCount2++) 
if  {pIMImaA->ppmcElem [ inCountl] [inCount2] . flRe  <  0) 

voSetCom ( & (pIMImaB->ppmcElem [ inCountl ] [ inCount2 ] ) , 0 , 0 ) ? 

else 

voSetCom ( & (pIMImaB->ppmcElem[ inCountl] [inCount2] ) ,pIMImaA- 
>ppmcElem[ inCountl] [inCount2] . flRe,0) ; 
break; 

case  EXP; 

for  ( inCount 1=0 ; inCount l<pIMImaB-> inNumRows ; inCountl++) 
for  ( inCount2=0; inCount2<pIMImaB->inNumCols ; inCount2++) 

voExpCom  ( ScpIMImaB->ppmcElem [  inCountl ]  [inCount2]  , pIMImaA- 
>ppmcElem[ inCountl] [inCount2] ) ; 
break; 

case  LOG: 

for  { inCountl=0 ; inCount l<pIMImaB-> inNumRows ; inCountl ++ ) 
for  { inCount2=0 ; inCount2<pIMImaB->inNumCols ; inCount2++) 

voLogCom( Sep IMImaB->ppmcElem[ inCountl]  [inCount2]  ,pIMImaA- 
>ppmcElem [ inCount 1 ] [inCount2] ) ; 
break; 

case  MAGNITUDE: 

for  { inCountl =0 ; inCountl<pIMImaB->inNumRows ; inCountl++) 

{ 

for  ( inCount2=0 ; inCount2<pIMImaB->inNumCols ; inCount2++) 

{ 

voMagCom { (float  *)  Sc flBuffer 1 , pIMImaA->ppmcElem [inCountl] [inCount2] } ; 
voSetCom (& (pIMImaB->ppmcElem [ inCountl]  [inCount2] ) #  flBuf ferl, 0) ; 

} 

} 

break; 

case  PHASE: 

for  (inCountl=0; inCount l<pIMImaB-> inNumRows ; inCountl++) 

{ 

for  ( inCount2=0 ; inCount2<pIMImaB->inNumCols ; inCount2++) 

{ 

voPhaCom (  (float  *)  ScflBuf ferl ( pIMImaA->ppmcElem[ inCountl]  [inCount2]  )  ; 
voSetCom ( Sc  (pIMImaB->ppmcElem[ inCountl] [inCount2] ) #  flBufferl.O) ; 

} 

} 

break; 

default: 

inStatus  =  NK; 
break; 

} 

return ( inStatus ) ; 

} 

- - 


- - 

int  in ImaS c aOp ( IMAGE  *pIMImaB,  IMAGE  *pIMImaA,  mcomplex  mcValue,  int  inOpType) 
{ 

int  inStatus, 

inCountl, 
inCount 2 ; 

inStatus  =  OK; 

switch  (inOpType) 

{ 

case  ADD: 

for  (inCount 1=0; inCount l<pIMImaB-> inNumRows ; inCountl++) 
for  ( inCount2=0 ; inCount2<pIMImaB->inNumCols ; inCount2++) 


voAddCom (&pIMImaB->ppmcElem [ inCount 1 ] [inCount2] ,pIMImaA- 
>ppmcElem [ inCountl ] [inCount2] ,mcValue) ; 
break; 

case  SUB: 

for  ( inCountl=0 ; inCountl<pIMImaB->inNumRows ; inCountl++) 
for  ( inCount2=0 ; inCount2<pIMImaB->inNumCols ; inCount2++ ) 

voSubCom { &pIMImaB->ppmcElem [ inCountl ] [inCount2] ,pIMImaA- 
>ppmcElein[  inCountl]  [inCount2]  ,mcValue)  ; 
break; 

case  MUL: 

for  ( inCountl =0 ; inCount l<pIMImaB->inNumRows ; inCountl++) 
for  ( inCount2=0 ; inCount2<pIMImaB->inNumCols ; inCount2++ ) 

voMulCom  ( &pIMImaB->ppmcElem [ inCountl ] [ inCount 2 ] , pIMImaA- 
>ppmcElem[ inCountl] [inCount2] ,mcValue) ; 
break; 

case  DIV: 

for  ( inCountl =0 ; inCount l<pIMImaB->inNumKows ; inCountl++ ) 
for  (inCount2=0 ; inCount2<pIMImaB->inNumCols ; inCount2++) 

voDivCom { &pIMImaB->ppmcElem [ inCountl ] [inCount2] ,pIMImaA- 
>ppmcElem[ inCountl] [inCount2] ,mcValue) ; 
break; 

case  POW: 

for  < inCountl =0 ; inCountl<pIMImaB->inNumRows ; inCountl++) 
for  ( inCount2=0 ; inCount 2 <pIMImaB->inNumCo Is ; inCount2++) 
voPowCom ( &pIMImaB->ppmcElem [ inCountl] [inCount2] ,pIMImaA- 
>ppmcElem [ inCountl ] [inCount 2] ymcValue) ; 
break; 

default : 

inStatus  =  NK; 
break; 

} 

return (inStatus) ; 

} 

- - 


- - 

int  inImaMatOp( IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inOpType) 

{ 

int  inStatus, 

inCountl, 
inCount2 ; 


inStatus  =  OK; 

switch  { inOpType ) 

{ 

case  ADD: 

for  ( inCount 1=0 ; inCountl<pIMImaC->inNumRows ; inCountl++) 
for  ( inCount2=0 ; inCount2<pIMImaC->inNumCols ; inCount2++) 
voAddCom ( &pIMImaC->ppmcElem [ inCountl ] [inCount 2] , 

pIMImaA->ppmcElem[ inCountl] [inCount2] , 

pIMImaB->ppmcElem [ inCountl ] [ inCount2 ] ) ; 

break; 
case  SUB: 

for  ( inCount 1=0 ; inCount l<pIMImaC->inNumRows ; inCount 1++ ) 
for  { inCount2=0 ; inCount2<pIMImaC->inNumCols ; inCount2++) 

voSubCom (&pIMImaC->ppmcElem[ inCountl] [inCount2] , 

pIMImaA->ppmcElem[ inCountl] [inCount2] , 
pIMImaB->ppmcElem[ inCountl] [inCount2] ) ; 

break  ,- 
case  MUL: 

for  ( inCount 1=0 ; inCount l<pIMImaC->inNumRows ; inCountl++) 
for  ( inCount2 =0 ; inCount2<pIMImaC->inNumCols ; inCount2  ++ ) 

voMulCom ( &pIMImaC->ppmcElem [ inCountl ] [inCount2], 

pIMImaA->ppmcElem [ inCountl ] [inCount2] , 
pIMImaB->ppmcElem[ inCountl] [inCount2] ) ; 
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break; 


case  DIV: 

for  { inCount  1=0 ;  inCount l<pIMImaC~>inNumRows ;  inCountl ++) 
for  { inCount 2=0 ; inCount2<pIMImaC->inNumCols ; inCount2++) 

voDivCom { &pIMImaC->ppmcElem [ inCountl ] [inCount2] # 

pIMImaA->ppmcElem[inCountl] [inCount2] , 
pIMImaB->ppmcElem[ inCountl] [inCount2] ) ; 

break; 

case  POW: 

for  ( inCount 1=0 ; inCount l<pIMImaC->inNumRows ; inCountl++) 
for  { inCount2=0 ; inCount2<pIMImaC->inNumCols ; inCount2++) 
voPowCom ( &pIMImaC->ppmcElem [ inCountl ] [inCount2] , 

pIMImaA->ppmcElem[ inCountl] [inCount2] , 
pIMImaB->ppmcElem[ inCountl] [inCount2] ) ; 

break; 

default : 

inStatus  =  NK; 
break ; 


return (inStatus) ; 

} 


//- 


int  inDFTIma( IMAGE  *pIMOutIma,  IMAGE  *pIMInIma/  int  inTransformType) 

{ 

inStatus, 

inNumElem, 
inCountl, 
inCount2 , 
inBasel, 
inBase2 ; 

unsigned  long  *pulDims; 
float  *pf iData; 

inStatus  =  OK; 

inNumElem  =  ( 2 *pIMInIma->inNumRows*pIMInIma->inNumCols ) ; 

pf IData  =  (float  *)  calloc (inNumElem, sizeof (float) ) ; 

for  (inCount 1=0 ; inCount l<pIMInIma->inNumRows; inCountl++) 

inBasel  =  inCountl* (2 *pIMInIma->inNumCols ) ; 

for  (inCount2=0 ; inCount2<pIMInIma->inNumCols ; inCount2++) 

{ 

inBase2  =  inBasel+(2*inCount2); 

pf IData [inBase2 ]  =  pIMInIma->ppmcElem[ inCountl] [inCount2] . f IRe; 
pf IData [inBase2+l]  =  pXMInIma->ppmcElem [ inCountl] [inCount2 ) . f llm; 

} 

} 

pulDims  =  (unsigned  long  *)  calloc (2 , sizeof (unsigned  long) ) ; 

pulDims [ 0 ]  =  pIMInIma->inNumRows ; 
pulDims [1]  =  pIMInIma->inNumCols; 

voDFTNDFT (pf IData, pulDims, 2 , inTransformType) ; 

for  { inCount 1= 0 ; inCountl<pIMOutIma->inNumRows ? inCountl++) 

inBasel  =  inCountl* (2 *pIMOutIma->inNumCols ) ; 

for  (inCount2=0 ; inCount2<pIMOutIma->inNumCols ; inCount2++) 

{ 

inBase2  =  inBasel+(2*inCount2); 

voSetCom(& (pIMOutIma->ppmcElem [ inCountl]  [inCount2] ) ,pf IData [ inBase2 ] ,pf IData [inBase2  +  l] )  ; 
} 

} 

free (pf IData)  ; 
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free (pulDims ) ; 
return ( inStatus ) ; 

} 

U - 


// -  * 

int  inZeroPadlmage < IMAGE  *pIMOutIma,  IMAGE  *pIMInIma,  int  inType) 

{ 

int  inStatus, 

inCountl, 
inCount2 , 
inRowPos, 
inColPos , 
inRowFirSeg, 
inRowSecSeg, 
inColFirSeg, 
inColSecSeg; 

IMAGE  * pIMImal; 

inStatus  =  OK; 

for  (inCount 1=0; inCount l<p IMOut Ima -> inNumRows ; inCount 1++) 
for  ( inCount2  =  0 ; inCount 2 <pIM0ut Ima ->inNumCo Is ; inCount2++) 
voSetCom (& ( p IMOut Ima - >ppmcElem [ inCountl] [inCount2]) ,0,0) ; 

switch  (inType) 

{ 

case  CORNERED: 

inlnsert Ima (pIMOutIma,pIMInIma,  0,0); 
break; 

case  CENTERED: 

inRowPos  =  (  (int)  floor  {  (  (float)  (pIMOutIma->inNumRows-pIMInIma->inNumRows )  )  12)  )  ; 
inColPos  =  ( (int) floor ( ( (float) (pIMOutIma->inNumCols-pIMInIma->inNumCols) ) /2) ) ; 
inlnsert Ima (pIMOutlma , pIMInlma , inRowPos , inColPos ) ; 
break; 

case  SPREAD: 

inRowFirSeg  =  ( (int) ceil (( { float )pIMInIma-> inNumRows) 12)  ) ; 

inRowSecSeg  =  pIMInIma->  inNumRows- inRowFirSeg; 
inColFirSeg  =  (( int) ceil ((( float) pIMInIma->inNumCols) /2 )) ; 
inColSecSeg  =  pIMInIma->inNumCols-inColFirSeg; 

pIMImal  =  pIMAsklmaMem ( inRowFirSeg, inColFirSeg) ; 

inExtractlma (pIMImal , pIMInlma, 0, 0, inRowFirSeg, inColFirSeg) ; 
inlnser tlma (pIMOutlma , pIMImal ,0,0); 
inFreelmaMem (pIMImal) ; 

pIMImal  =  pIMAsklmaMem (inRowSecSeg, inColFirSeg) ; 

inExtractlma (pIMImal, pIMInlma,  inRowFirSeg, 0 , inRowSecSeg, inColFirSeg) ; 
inlnsertlma  (pIMOutlma, pIMImal  ,pIMOutIma->inNumRows-inRowSecSeg,  0 )  ; 
inFreelmaMem (pIMImal) ; 

pIMImal  =  pIMAskImaMem(inRowFirSeg,inColSecSeg); 

inExtractlma (pIMImal, pIMInlma, 0,  inColFirSeg, inRowFirSeg, inColSecSeg) ; 
inlnsertlma  (pIMOutlma, pIMImal,  0 , pIMOutIma->inNumCols-inColSecSeg)  ; 
inFreelmaMem (pIMImal ) ; 

pIMImal  =  pIMAskImaMem(inRowSecSeg,inColSecSeg); 

inExtractlma  (pIMImal,  pIMInlma,  inRowFirSeg,  inColFirSeg,  inRowSecSeg,  inColSecSeg)  ; 
inlnsertlma  (pIMOutlma,  pIMImal,  pIMOutIma->inNumRows-inRowSecSeg,  pIMOutIma->inNumCols- 
inColSecSeg) ; 

inFreelmaMem (pIMImal) ; 

break; 

} 

return ( inStatus) ; 


// 
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int  inlmaConvolution ( IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inConType) 

{ 

int  inStatus, 

inTotRow, 

inTotCol, 

inExtRow, 

inExtCoI; 

IMAGE  *pIMBufferl, 

*pIMBuf  f er2 , 

*pIMBuffer3; 

inStatus  =  OK; 
if  (inConType  ==  LINEAR) 

inTotRow  =  pIMImaA->inNumRows+pIMImaB->inNumRows-l; 
inTotCol  =  pIMImaA->inNumCols+pIMImaB->inNumCols-l; 

pIMBufferl  =  pIMAsklmaMem (inTotRow, inTotCol) ; 
pIMBuf f er2  =  pIMAsklmaMem ( inTotRow, inTotCol ) ; 
pIMBuf f er3  =  pIMAsklmaMem (inTotRow, inTotCol) ; 

inZeroPadlmage (pIMBuf ferl, pIMImaA, CORNERED) ; 
inZeroPadlmage (pIMBuf  f er2 , pIMImaB, CORNERED) ; 

inDFTIma (pIMBuf fer3 , pIMBufferl , DFT) ? 

ini MAC opy Ima (pIMBuf ferl, pIMBuf fer3 ) ? 

inDFTIma (pIMBuf fer3 , pIMBuf fer2 , DFT) ; 

inIMACopylma (pIMBuf fer2 , pIMBuf fer3 ) ? 

inlmaMatOp (pIMBuf f er3 , pIMBuf ferl, pIMBuf fer2 ,MUL) ; 

inDFTIma (pIMBuf ferl , pIMBuf fer3 , IDFT) ; 

inExtRow  =  ( (int) floor ( ( (float) (pIMBuf ferl ->inNumRows-pIMImaC->inNumRows) ) /2) ) ; 
inExtCoI  =  ( (int) floor ( ( (float) (pIMBuf ferl->inNumCols-pIMImaC->inNumCols) ) / 2 ) ) ; 

inExtractlma (pIMBuf fer2 , pIMBufferl , inExtRow, inExtCoI , pIMImaC->inNumRows , pIMImaC->inNumCols) 
inlmaShift (pIMImaC, pIMBuf fer2) ; 

inFreelmaMem (pIMBuf ferl ) ; 
inFreelmaMem (pIMBuf fer2 ) ; 
inFreelmaMem (pIMBuf fer3) ; 

} 

else  if  (inConType  ==  CIRCULAR) 

pIMBufferl  =  pIMAsklmaMem (pIMImaA->inNumRows, pIMImaA->inNumCols ) ; 
pIMBuf fer2  =  pIMAsklmaMem ( pIMImaB- >inNumRows , pIMImaB->inNumCols ) ; 
pIMBuf  fer3  =  pIMAsklmaMem  (pIMImaC- >inNumRows,p IMImaC-> inNximCols )  ; 

inDFTIma (pIMBuf ferl , pIMImaA, DFT) ; 
inDFTIma (pIMBuf fer2 , pIMImaB, DFT) ; 

inlmaMatOp (pIMBuf fer3 , pIMBuf ferl, pIMBuf fer2 , MUL) ; 
inDFTIma (pIMImaC, pIMBuf fer3, IDFT) ; 

inFreelmaMem (pIMBuf ferl ) ; 
inFreelmaMem (pIMBuf fer2) ; 
inFreelmaMem (pIMBuf fer3) ; 

-  } 
else 
{ 

inStatus  =  NK; 

} 

return ( inStatus) ; 


// 


int  inlmaCorrelation ( IMAGE  *pIMImaC,  IMAGE  *pIMImaA,  IMAGE  *pIMImaB,  int  inCorType) 

{ 

int  inStatus; 

IMAGE  *pIMBuf f erl , 

*pIMBuffer2; 


inStatus  =  OK;  _  .  . 

pIMBufferl  =  pIMAsklmaMem  (pIMImaA->inNumHows ,  pIMImaA->inNumCols )  ; 
pIMBuf fer2  =  pIMAsklmaMem  (pIMImaA-> inNumRows ,  pIMImaA->inNumCols )  ; 
inHorFlipIma (pIMBufferl , pIMImaA) ; 
inVerFlipIma (pIMBuf fer2 , pIMBufferl ) ; 
inFreelmaMem (pIMBufferl) ? 

inlmaConvolution (pIMImaC,pIMImaB, pIMBuf fer 2 , inCorType) ; 
inFreelmaMem (pIMBuf fer2 ) ; 

return (inStatus) ; 

} 

// - 


// - 


int  inlmaShift (IMAGE  *pIMImaOut,  IMAGE  *pIMImaIn) 

inStatus, 

inRowFirSeg, 
inRowSecSeg, 
inColFirSeg, 
inColSecSeg; 

*pIMImal; 


{ 

int 


IMAGE 


inStatus  =  OK; 

inRowFirSeg  =  ( (int) ceil (( (float ) pIMImaIn->inNumRows ) /2) ) ; 
inRowSecSeg  =  pIMImaIn->inNumRows -inRowFirSeg; 
inColFirSeg  =  ( (int ) ceil ((( float ) pIMImaIn->inNumCols) /2 )} ; 
inColSecSeg  =  pIMImaIn->inNumCols-inColFirSeg; 

pIMImal  =  pIMAsklmaMem ( inRowFirSeg, inColFirSeg) ; 
inExtractlma (pIMImal , pIMImaln, 0 , 0 , inRowFirSeg, inColFirSeg) ; 

inlnsertlma (pIMImaOut, pIMImal, pIMImaOut->inNumRows- inRowFirSeg, pIMImaOut->inNumCols- 
inColFirSeg) ; 

inFreelmaMem (pIMImal ) ; 

pIMImal  =  pIMAsklmaMem (inRowSecSeg, inColFirSeg) ; 

inExtractlma (pIMImal , pIMImaln, inRowFirSeg, 0 , inRowSecSeg, inColFirSeg) ; 
inlnsertlma (pIMImaOut , pIMImal , 0 , pIMImaOut->inNumCols- inColFirSeg ) ; 
inFreelmaMem (pIMImal) ; 

pIMImal  =  pIMAsklmaMem (inRowFirSeg, inColSecSeg) ? 

inExtractlma (pIMImal, pIMImaln, 0, inColFirSeg, inRowFirSeg, inColSecSeg) ; 
inlnsertlma (pIMImaOut , pIMImal , pIMImaOut->inNumRows- inRowFirSeg, 0 ) ; 
inFreelmaMem (pIMImal) ; 

pIMImal  =  pIMAsklmaMem (inRowSecSeg, inColSecSeg) ; 

inExtractlma (pIMImal , pIMImaln, inRowFirSeg, inColFirSeg, inRowSecSeg, inColSecSeg) ; 
inlnsertlma (pIMImaOut , pIMImal ,0,0) ; 
inFreelmaMem (pIMImal) ; 

return (inStatus) ; 


it - 

int  inOTFFromAperture ( IMAGE  *pIMOTF,  IMAGE  *pIMAperture ,  int  inConType) 

{ 

int  inStatus, 

inCount 1 , 
inCount 2 ; 

float  flMax; 

mcomplex  mcValue; 

IMAGE  *pIMBuf ferl , 

*pIMBuf fer 2 ; 


inStatus  =  OK; 

pIMBufferl  =  pIMAsklmaMem ( pIMAper ture- > inNumRows ,  pIMAper ture- > inNumCols )  ; 
pIMBuf fer2  =  pIMAsklmaMem ( pIMAper ture -> inNumRows , pIMAperture->inNumCols ) ; 
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inCopy Ima (pIMBuf f erl , pIMAperture) ; 

inlmaCorrelation (pIMBuf fer2 , pIMAperture , pIMBuf f erl , inConType) ; 
inlmaFunOp (pIMBuf ferl , pIMBuf fer2 , IMRA) ; 

flMax  =  0; 

for  ( inCountl=0 ;  inCountl<pIMBuf  f  erl->inNuxnRows  ?  inCountl++ ) 

for  (inCount2=0; inCount2<pIMBufferl~>inNumCols;inCount2++) 

inlMAGetlmaElemC&mcValue, pIMBuf ferl, inCount 1, inCount2) ; 
if  (flMax  <  me Value . flRe) 
flMax  =  mcValue. f IRe; 

> 

} 

voSetCom(&mc Value, flMax, 0) ? 

inlmaScaOp (pIMOTF , pIMBuf  ferl , mcValue , DIV) ; 

inFreelmaMem (pIMBuf ferl ) ; 
inFreelmaMem (pIMBuf fer2) ; 

return (inStatus) ; 


//  — 


- - 

int  inImaMetric( float  *pf IDistance,  IMAGE  *pIMA,  IMAGE  *pIMB,  int  inMetricType,  float  flParam) 

{ 

int  inStatus, 

inCount 1, 
inCount2 ; 

float  flBuffer, 

doNorm ; 

mcomplex  mcMeanA, 

mcMeanB , 
mcVarA, 
mcVarB, 
mcBuf ferl, 
mcBuf fer2 , 
mcCrossCorr; 

inStatus  =  OK; 

switch  (inMetricType) 

( 

case  LPNORM: 

flBuffer  =  0; 

for  ( inCount 1 = 0 ; inCount l<pIMA->inNumRows ; inCountl++) 

for  (inCount2=0; inCount2<pIMA->inNumCols ; inCount2++) 

{ 

voSubCom ( fcmcBuf ferl, pIMA->ppmcElem[ inCount 1) [inCount23 ,pIMB- 
>ppmcElem [ inCountl ] [inCount2] ) ; 

voMagCom(&idoNonn,  mcBuf  ferl)  ; 

flBuffer  +=  (f loat)pow( doNorm, flParam) ? 

} 

( *pf IDistance)  =  ( float) pow( flBuffer/ (pIMA->inNumRows*pIMA->inNumCols) , 1/flParam) ; 
break; 

case  CORRELATION: 
inlmaMean (&mcMeanA, pIMA) ; 
inlmaMean ( &mcMeanB , pIMB ) ; 

inlmaVar ( fcmcVarA, pIMA) ; 
inlmaVar (&mcVarB, pIMB) ; 

voSetCom ( imcCrossCorr ,0,0) ; 

for  {inCount 1=0; inCountl<pIMA->inNumRows ; inCountl++) 

{ 

for  (inCount2=0; inCount2<pIMA->inNumCols ; inCount2++) 

{ 

voConjugateCom(&mcBuf ferl, pIMB->ppmcElem[ inCountl] [inCount2] ) ; 
voMulCom(&mcBuf fer 2, pIMA- >ppmcElem[ inCountl]  [inCount2]  ,mcBufferl)  ; 
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voAssignCom (&mcBuf  ferl ,mcCrossCorr )  ; 
voAddCom(&mcCrossCorr, mcBuf ferl, mcBuf fer2 ) ; 
} 


voAssignCom (&mcBuf ferl ,mcCrossCorr) ; 

voSetCom (&mcBuf f er2 , (float) (pIMA->inNumRows*pIMA->inNumCols) # 0 ) ; 
voDivCom  (&mcCrossCorr ,  mcBuf  ferl, mcBuf  fer2  )  ; 

voConjugateCom (&mcBuf ferl, mcMeanB) ; 
voMulCom (&mcBuf fer2  ,mcMeanA, mcBuf ferl) ; 
voAssignCom (&mcBuf ferl ,mcCrossCorr)  ; 
voSubCom ( kmcCrossCorr , mcBuf ferl , mcBuf  f er2 ) ; 

voMagCom  ( &f  lBuf  f  er ,  mcCrossCorr )  *; 

( *pf IDistance)  =  flBuffer; 
voMagCom (&f lBuf fer ,mcVarA) ; 
if  (flBuffer  ==  0) 

(*pf IDistance)  =  0; 
else 

{ *pf IDistance)  /=  (float)sqrt(flBuffer); 
voMagCom (& flBuffer , mcVarB) ; 
if  (flBuffer  ==  0) 

( *pf IDistance)  =  0; 
else 

{ *pf IDistance)  /=  ( float ) sqrt ( flBuffer) ; 

break ; 

default : 

inStatus  =  NK; 

break; 


return ( inStatus ) ; 

) 


int  inSimMap ( IMAGE  *pIMMap,  IMAGE  *pIMRef/  IMAGE  *pIMIma,  int  inBlockSize,  int  inMetricType , 
float  flParam) 

{ 

int  inStatus, 

inCount 1 , 
inCount2 , 
inCount3 , 
inCount4 ; 

float  f lValue; 

IMAGE  *pIMBuf f erl , 

*pIMBuf f er2 ; 


inStatus  =  OK; 

pIMBufferl  =  pIMIMAAsklmaMem ( inBlockSize, inBlockSize) ; 
pIMBuffer2  =  pIMIMAAsklmaMem (inBlockSize, inBlockSize) ; 

for  (inCountl=0; inCount l<pIMRe f->inNumRows ; inCountl+=inBlockSize) 

{ 

for  (inCount2=0; inCount 2 <pIMRef ->inNumCols ; inCount2+=inBlockSize) 

{ 

inExtractlma (pIMBuf ferl,pIMRef , inCountl, inCount2 , inBlockSize, inBlockSize) ; 
inExtractlma (pIMBuf fer2 , pIMIma, inCountl, inCount2 , inBlockSize, inBlockSize) ; 

inlmaMetric (&f lvalue, pIMBuf ferl, pIMBuf fer2 , inMetricType, f lParam) ; 

for  (inCount3=inCountl; inCount3<inCountl+inBlockSize ; inCount3++) 

{ 

for  (inCount 4= inCount 2  ;  inCount4<inCount2+inBlockSize;  inCount4-»-+) 

{ 

voSetCom (&  (pIMMap“>ppmcElem[inCount3  3  [inCount4])  ,  f  lvalue,  0)  ; 

} 

} 

} 
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} 


inIMAFreelmaMem (pIMBuf ferl ) ; 
inIMAFreelmaMem (pIMBuf fer2 ) ; 

return ( inStatus )  ? 

} 

// - 

// - 


int  inGetWindow( IMAGE  *pIMWindow,  int  inWindowType) 

{ 

int  inStatus, 

inCount 1, 
inCount 2; 

float  flRowCen, 

f IColCen, 
flCenRow, 
flCenCol, 
f 1 Ext Row, 
fl Ext Col, 
f IParLength, 
f ITotLength, 
flPos; 


inStatus  =  OK; 

//  CACA 

flRowCen  =  { float )  (pIMWindow->inNumRows /2 )  ; 
f  IColCen  =  (float) (pIMWindow->inNumCols/2 ) ; 

for  (inCountl=0; inCount 1 <p IMWindow- > inNumRows ; inCountl++) 

( 

for  { inCount2=0 ; inCount 2 <pIMWindow->inNumCo Is ; inCount2++) 

( 

flCenCol  =  {( float) inCount2 ) -f IColCen; 
f ICenRow  =  {( float ) inCountl ) -flRowCen; 

if  {(flCenCol  ==  0)  &&  (flCenRow  ==  0)) 

{ 

flPos  =  0.5; 

} 

else 

{ 

if  (flCenCol  ==  0) 

{ 

f lExtCol  =  0; 
f lExtRow  =  flRowCen; 

} 

else 

{ 

f lExtCol  =  f IColCen ; 

f lExtRow  =  (f loat) fabs ( (flCenRow/f lCenCol) *f IColCen) ; 

if  (f lExtRow  >  flRowCen) 

{ 

f lExtCol  =  (float) fabs ( (flCenCol /flCenRow) * flRowCen) ; 
f lExtRow  =  flRowCen; 

} 

} 

f IParLength  =  ( float) sqrt ( f lCenCol * flCenCol + flCenRow* f ICenRow) ; 

flTotLength  =  (float) (2*sqrt ( f lExtCol*f lExtCol+f lExtRow*f lExtRow) ) ; 

flPos  =  (f lTotLength/2+f IParLength) /flTotLength; 

) 

switch  { inWindowType ) 

{ 

case  RECTANGULAR: 

voSetCom { & (pIMWindow->ppmcElem [ inCountl ] [ inCount2 ) ) , 1 , 0 ) ; 
break; 

case  HAMMING: 
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voSetCom (& (pIMWindow->ppmcElem[ inCountl] [inCount2]) , (float) (0.54 
0 . 46*cos (2*PI*flPos) ) ,0) ; 
break; 

case  HANNING:  .  ...  /n  cn 

voSetCom  (&  (pIMWindow->ppmcElem[  inCountl]  [mCount2]  )  ,  (float)  (0.50- 

0 . 50*cos (2*PI*f IPos) ) , 0) ; 
break; 


case  BLACKMAN: 

voSetCom (& (pIMWindow->ppmcElem[ inCountl] [inCount2]) , (float) (0.42 
0 . 50*cos (2*PI*flPos)+0. 08*cos (4*PI*flPos) ) ,0) ; 
break; 


default : 

inStatus  =  NK; 
break ; 

} 

} 

} 

return (inStatus) ; 

} 

// - 


// - 


int  inLikelihOOd( float  ‘pflLik,  IMAGE  *pIMg,  IMAGE  ‘pIMEstBlur ,  IMAGE  *pIMf, 

IMAGE  *pIMPrior ,  int  inType) 


{ 

int 


float 


inStatus, 

inCountl , 
inCount2 ; 
f lBuf ferl , 

fig, 
f  leb, 
flf, 
fip; 


inStatus  =  OK; 

switch  (inType) 

{ 

case  ML: 

flBufferl  =  0; 

for  ( inCount 1=0 ; inCountl<pIMg->inNumRows ; inCountl ++) 

for  ( inCount2=0; inCount2<pIMg->inNumCols ; inCount2++) 

{ 

fig  =  pIMg->ppmcElem [inCountl ] [inCount2] . flRe+1 ; 

fleb  =  pIMEs tBlur->ppmcElem[ inCountl ] [inCount2] . flRe+1; 

flBufferl  +=  (fig* ( (float) log (fleb) ) -f leb-f lLogNatFac (fig) ) ; 

> 

} 

( *pf ILik)  =  flBufferl; 

( *pf lLik)  /=  (pIMg->inNumRows*pIMg->inNumCols) ; 
break; 

case  MAP: 

flBufferl  =  0; 

for  ( inCount 1=0 ; inCountl<pIMg->inNumRows ; inCountl++) 

for  ( inCount2=0 ; inCount2<pIMg->inNumCols ; inCount2++) 

fig  =  pIMg->ppmcElem [inCountl ] [inCount2] . flRe+1; 

fleb  =  pIMEs tBlur->ppmcElem[ inCountl 3 [inCount2] . flRe+1; 

flBufferl  +=  (fig* ( (float) log ( fleb) ) -f leb-f lLogNatFac (fig) ) ; 

} 

} 

(*pf ILik)  =  flBufferl; 
flBufferl  =  0; 

for  ( inCountl=0 ; inCount l<pIMf ->inNumRows ; inCountl++ ) 

{ 
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for  (inCount2=0; inCount 2 <pIMf -> inNumCol s ; inCount2++) 

{ 

flf  =  pIMg->ppmcElem[inCountl] [inCount2] . flRe+1; 

flp  =  pIMEstBlur->ppmcElem [inCountl 3 [inCount2] . flRe+1; 

flBufferl  +=  (flf*{ (float) log(flp) ) -f lp-f lLogNatFac { f If ) ) ; 

} 

) 

( *pf lLik)  +  =  flBufferl; 

(*pflLik)  /=  (pIMg->inNumRows*pIMg->inNumCols )  ; 
break ; 

default : 

inStatus  =  NK; 

( *pf ILik)  =  0; 
break; 

} 

return (inStatus)  ; 

} 

// - 


If 


float  flLogNatFac 
{ 

float 

float 


(float  flNumber) 

flCoef f icients  £  6 ] ; 
flBufferl, 

f lBuf fer2 , 
f lBuf fer3 , 
f lBuf f er4 ; 


flCoef f icients [0] 
flCoeff icients [1] 
flCoeff icients [2 ) 
flCoeff icients [3 ] 
flCoeff icients [4 ] 
flCoeff icients [ 5 ] 


(float) (+76.180091729471460000) 
(float)  (-86.505320329416770000) 
(float) (+24.014098240830910000) 
(float) (-01.231739572450155000) 
(float) (+00.001208650973866179) 
(float) (-00.000005395239384953) 


flBufferl 
f lBuf fer2 
f lBuf fer3 
f lBuf fer4 
f lBuf f er4 
f lBuf fer4 
f lBuf f er4 
f lBuf f er4 
f lBuf fer4 
f lBuf fer4 


=  flNumber+1; 

=  flBufferl; 

=  (float) (f lBuf ferl+5 . 5- ( f lBuf ferl+0 . 5) *log (f lBuf f erl+5 . 5 ) ) ; 
=  (float)  (1.000000000190015)  ; 

+=  flCoef f icients [0] /++flBuf fer2 ; 

+=  flCoef f icients [1] /++flBuf fer2 ; 

+=  flCoef f icients [2] /++f lBuf fer2 ; 

+=  flCoeff icients [3 3 /++f lBuf fer2 ; 

+=  flCoeff icients [43 /++f lBuf fer2 ; 

+=  flCoeff icients [53 /++flBuf fer2 ; 


return ( { (float) log (2 . 5066282746310005*f lBuf f er4/f lBuf ferl) )-flBuffer3) ; 

} 


// 

// 


int  inTotalFlip (IMAGE  *pIMOutIma,  IMAGE  *pIMInIma) 
{ 

int  inStatus, 

inNurnRow, 

inNumCol; 

IMAGE  *pIMBuf f erl , 

*pIMBuffer2; 


inStatus  =  OK; 

inNurnRow  =  pIMInIma->inNumRows; 
inNumCol  =  pIMI nlma -> inNumCol s ; 

pIMBuf f erl  =  pi MI MAAskImaMem( inNurnRow, inNumCol) ; 
pIMBuf f er2  =  pIMIMAAskImaMem( inNurnRow, inNumCol) ; 

inlnser tlma (pIMOutlma , pIMInlma ,0,0); 
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inHorFlipIma (pIMBuf ferl,pIMInIma> ; 
inlnsertlma (pIMOutlma, pIMBuf ferl , inNumRow, 0 ) ; 

inVerFlipIma (pIMBuf ferl , pIMInlma) ; 
inlnsertlma (pIMOutlma, pIMBuf ferl , 0, inNumCol) ? 

inHorFlipIma (pIMBuf ferl , pIMInlma) ; 

inVerFlipIma (pIMBuf fer2# pIMBuf ferl) ; 

inlnsertlma (pIMOutlma , pIMBuf fer2  # inNumRow, inNumCol ) 

inFreelmaMem (pIMBuf ferl) ; 
inFreelmaMem (pIMBuf fer2) ; 

return (inStatus) ; 

} 
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NAME 

mcomplex. h 

DESCRIPTION 

Header  file  for  mcomplex. cpp 

PROGRAMMER 

Pablo  Zegers  _ 


// - 

#i fndef  Fmcomplex 

// - 

// - 

//  INCLUDE  FILES 

// - 

#include  "  .  .  \Global\Global  .h" 

1 - - 

//  STRUCTURE  DEFINITIONS 

// - 

typedef  struct 

{ 

float  flRe, 

film; 

}  mcomplex; 

- - 

//  PUBLIC  PROTOTYPES 


void  voSetCom (mcomplex  *pmcA,  float  flRe,  float  film) ; 

void  voAssignCom (mcomplex  *pmcB,  mcomplex  mcA) ; 

void  voConjugateCom (mcomplex  *pmcB,  mcomplex  mcA) ; 

void  voMagCom( float  *pflMag,  mcomplex  mcA) ; 

void  voPhaCom( float  *pflPhase,  mcomplex  mcA) ; 

void  voAddCom (mcomplex  *pmcC,  mcomplex  mcA,  mcomplex  mcB) ; 

void  vo SubCom (mcomplex  *pmcC/  mcomplex  mcA,  mcomplex  mcB) ; 

void  voMul Com (mcomplex  *pmcC#  mcomplex  mcA,  mcomplex  mcB) ; 

void  vo DivCom (mcomplex  *pmcC/  mcomplex  mcA,  mcomplex  mcB) ; 

void  voExpCom (mcomplex  *pmcB,  mcomplex  mcA) ; 

void  voLogCom (mcomplex  *pmcB/  mcomplex  me A) ; 

void  voPowCom (mcomplex  *pmcB,  mcomplex  mcA,  mcomplex  mcB) ; 

- - 

#define  Fmcomplex  0 

#endif 

- - 
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//  DESCRIPTION  :  Functions  to  manipulate  complex  numbers 
//  AUTHOR  :  Pablo  Zegers 

//  DATE  :  March,  1998 

// - 

// - 

//  INCLUDE  FILES 

// - - - 

# include  <stdlib.h> 

#include  <stdio.h> 

# include  <io.h> 

# include  <string.h> 

# include  <malloc.h> 

# include  <math.h> 

# include  "mcomplex.h" 

// - 

//  PUBLIC  ROUTINES 

- - 

If - 

// - 

void  voSetCom (mcomplex  *pmcA,  float  flRe,  float  film) 

{ 

pmcA->flRe  =  flRe; 
pmcA->flIm  =  film; 

} 

- - 

// - 

void  voAs si gnCom (mcomplex  *pmcB,  mcomplex  mcA) 

{ 

pmcB->flRe  =  mcA.flRe; 
pmcB->flIm  =  mcA.fllm; 

} 

// - 


void  voConjugateCom (mcomplex  *pmcB,  mcomplex  mcA) 

{ 

pmcB->flRe  =  mcA.flRe; 
pmcB->flIm  =  -mcA.fllm; 

} 

- - 

- - 

void  voMagCom{ float  *pflMag,  mcomplex  mcA) 

( *pf lMag)  =  ( float) sqrt (mcA. f lRe*mcA. f iRe+mcA. f Hm*mcA. film) ? 

} 

// - 

- - 

void  voPhaCom( float  *pflPhase,  mcomplex  mcA) 

{ 

(*pflPhase)  =  ( float) atan (mcA. fllm/mcA. flRe) ? 

} 

- - 

- - 

void  voAddCom (mcomplex  *pmcC,  mcomplex  mcA,  mcomplex  mcB) 

{ 

pmcC->f IRe  -  mcA. f lRe+mcB . f lRe ; 
pmcC->flIm  =  mcA.  f  Hm+mcB.  film; 

} 

- - 
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// 


void  voSubCom (mcomplex  *pmcC,  mcomplex  me A,  mcomplex  mcB) 

pmcC->f lRe  =  mcA. f IRe-mcB . f IRe; 
pmcC->flIm  =  mcA. fllm-mcB. film; 


n - - — mil — 

void  voMul Com (mcomplex  *pmcC,  mcomplex  mcA,  mcomplex  mcB) 

pmcC->f IRe  =  mcA. f IRe *mcB . flRe-mcA. flIm*mcB. film; 
pmcC->flIm  =  mcA. flRe*mcB. fllm+mcA. flIm*mcB. flRe; 


// - :::::::::::::::::: 

void  voDivCom (mcomplex  *pmcC,  mcomplex  mcA,  mcomplex  mcB) 

float  flMag2; 

pmcC->flRe  =  mcA. f IRe *mcB. flRe+mcA. flIm*mcB. film; 
pmcC->f 11m  =  mcA. flIm*mcB. flRe-mcA. f IRe *mcB. film; 

f  lMag2  =  mcB .  f  lRe*mcB .  f  IRe+mcB.  f  Hm*mcB.  f  11m; 

if  ( f lMag2  !=  0) 

{ 

pmcC->flRe  !-  flMag2; 
pmcC->flIm  /=  flMag2; 

} 

} 

- - 

- - 

void  voExpCom (mcomplex  *pmcB,  mcomplex  me A) 

pmcB->f IRe  =  { float) (exp (mcA. f IRe) *cos (mcA. film) ) ; 

pmcB->f 11m  =  (float) (exp (mcA.f IRe) *sin(mcA. film)); 


- - 

it - 

void  voLogCom (mcomplex  *pmcB,  mcomplex  mcA) 

{ 

float  flMag, 

f 1 Phase; 

flMag  =  ( float) sqrt (mcA.f lRe*mcA. flRe+mcA. flIm*mcA. film) ; 
if  (mcA.f IRe  !=  0) 

{ 

flPhase  =  ( float) atan (mcA. fllm/mcA. f IRe) ; 

} 

else 

{ 

if  (mcA. film  >  0) 

flPhase  =  ( float) (+PI/2 ) ; 
else 

flPhase  =  (float) (-PI/2) ; 

} 

pmcB->f IRe  =  ( float ){ log ( flMag) ) ; 
pmcB->f 11m  =  flPhase; 


// - 

- - 

void  vo PowCom (mcomplex  *pmcB/  mcomplex  mcA,  mcomplex  mcB) 

{ 

mcomplex  mcBufferl, 

mcBuf fer2 ; 

voLogCom  (&mcBufferl,  me  A)  ; 

voMul Com ( &mcBuf f er2 , mcB , mcBufferl ) ; 

voExpCom (pmcB , mcBuf fer 2 ) ; 
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